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I.  INTRODUCTION 


A.  GENERAL 

Over  the  past  15  years  the  reliance  on  software,  especially  within  critical 
weapons  systems,  has  grown  in  orders  of  magnitude.  Congressional  and  other 
governmental  reports  during  this  same  period,  however,  consistently  cite  the 
lack  of  success  of  the  Department  of  Defense  in  contracting  for  the  develop¬ 
ment  of  specialized  software.  Unfortimately,  the  flexibility  offered  by  using 
software  as  an  integral  part  of  a  weapon  is  often  stifled  by  the  acquisition 
system  used  to  procure  il.  Tlie  Department  of  Defense's  ability  to  contract  for 
the  development  of  this  mission  critical  software  directly  affects  the  future 
defense  posture  of  our  Nation. 

B.  AREA  OF  RESEARCH  AND  OBJECTIVE 

This  thps’P  examines  the  contracting  and  management  guidance  cur¬ 
rently  in  place  regarding  the  procurement  of  custom  developed  software 
within  the  Department  of  the  Navy.  The  object  of  this  research  is  to  explore 
options  available  to  program  managers  and  con^^rarting  officers  that  may 
improve  the  success  of  acquisition  of  embedded  software. 

C.  RESEARCH  QUESTIONS 

1.  Primary  Question 

What  currently  allowable  contractual  mechanisms  can  be  used  to 
improve  the  Department  of  the  Navy's  success  in  the  procurement  of  custom 
developed  software  ? 
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2.  Subsidiary  Questions 

a.  How  successful  is  the  Department  of  Defense/Department  of 
the  Navy  in  the  procurement  of  custom  software? 

b.  What  directives  regulate  and'or  guide  Program  Managers  and 
Contracting  Officers  in  the  procurement  of  custom  software  ? 

c.  What  contract  types  are  typically  used  for  programs  involving 
the  development  of  custom  software  ? 

d.  Are  there  any  contractual  mechanisms  that  can  be  used  to  aid 
in  the  development  of  custom  software  ? 

D.  SCOPE 

This  thesis  investigates  the  contractual  mechanisms  and  strategies  cur¬ 
rently  allowed  by  acquisition  laws  and  regulations  that  may  assist  program 
managers  and  contracting  officers  in  the  procurement  of  mission  critical  soft¬ 
ware  for  weapon  systems.  The  term  "mission  critical  software  for  weapons 
systems"  refers  to  embedded  computer  programs  whose  failure  to  perform 
properly  may  result  in  loss  of  the  weapon  system,  loss  of  a  human  being,  loss 
of  mission  capability,  or  severe  personal  injury.  Mission  critical  software 
excludes  all  administrative  type  computer  programs. 

Although  the  laws  and  regulations  apply  across  the  hoard  for  all  Federal 
agencies,  this  thesis,  in  most  cases,  deals  within  the  confines  of  the  Depart¬ 
ment  of  the  Navy.  It  focuses  on  contractual  types,  tools,  mechanisms  and 
strategies  that  have  been  used  in  the  past  during  software  development  pro¬ 
grams.  The  results  of  this  research  are  directed  towards  Department  of  the 
Navy  program  managers  and  contracting  officers.  For  the  purposes  of  this 
research,  the  success  of  a  program  is  based  on  the  Department  of  the  Navy's 
ability  to  deliver  a  quality  weapon  system  to  the  fleet,  that  is — mission 
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capable,  on  time,  on  budget.  Simplifying  it  one  step  further  by  taking  a  "user- 
based"  approach  to  quality,  success  is  measured  by  how  a  product  satisfies 
the  user's  needs  [Ref.  1]  . 

This  thesis  takes  a  strict  managerial  view  of  software  development.  It 
does  not  address  specific  technical  issues.  It  is  the  program  manager  and  con¬ 
tracting  officer  who  must  bridge  the  gap  between  the  "ones  and  zeros" 
programmers  and  the  end  user.  With  that  in  mind,  this  thesis  explores  the 
tools  available  to  accomplish  such  a  critical  task. 

E.  METHODOLOGY 

1.  Literature  Search 

A  literature  search  was  condi  .  d  in  order  to  validate  the  basic 
premise  of  the  thesis,  that  in  fact  ther-  *s  a  problem  regarding  the  procure¬ 
ment  of  custom  developed  softw'are  '  the  Department  of  Defense.  Sources  of 
this  literature  ’•<'view  were  Genera  Accounting  Office  audit  reports  and  the 
Defense  Logistics  Studies  Information  Exchange.  Extensive  use  was  made  of 
the  Naval  Postgraduate  School  Library,  a  university  library  and  information 
center  featuring  an  on-line  bibliographic  search  capability.  A  review  of  both 
government  and  non-govemment  publications  and  periodicals  was  conducted. 

2.  Survey 

A  written  questionnaire  was  sent  to  150  government  and  industry 
professionals  with  various  backgrounds  including  program  management,  sys¬ 
tem  engineering,  contracting  and  contract  administration,  test  and  evalua¬ 
tion,  and  quality  assurance.  A  copy  of  the  questionnaire  used  during  the 
survey  is  contained  in  Appendix  A.  The  results  of  this  data  collection  effort 
are  contained  in  Chapter  V  and  Appendix  B. 
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3.  Interview 

Personal  and  telephone  interviews  were  conducted  with  Govern¬ 
ment  and  non-govemment  personnel  with  varying  background  experiences. 
Although  the  number  of  interviews  conducted  was  small,  an  attempt  was 
made  to  interview  a  broad  spectrum  of  people  involved  in  the  software  devel¬ 
opment  process.  The  areas  of  expertise  of  those  persons  interviewed  riinged 
from  corporate  vice-president,  to  contracting  officer,  to  software  quality 
assurance  representative,  to  independent  validation  and  verification  person¬ 
nel.  The  intent  of  the  interviews  was  to  supplement  and  amplify  that  cata 
collected  through  the  survey.  A  list  of  the  people  interviewed  is  containec  in 
Appendix  C. 

F.  LIMITATIONS 

Although  65  of  the  150  questionnaires  distributed  were  returned 
(a  response  rat ;  of  43%)  only  49  had  had  experience  in  software  development 
and  acquisition.  First  hand  information  regarding  specific  weapon  system 
programs  was  difficult  to  obtain.  Most  of  the  data  concerning  the  problems 
encountered  during  the  procurement  of  embedded  mission  critical  software 
were  obtained  through  second  sources,  such  as  DoD  and  General  Accounting 
Office  (GAO)  reports.  Reliance  on  these  types  of  reports,  in  lieu  of  data 
directly  from  the  program  offices,  may  sometimes  lead  to  generalizations  of 
the  entire  weapon  system  program  based  on  problem  areas  in  software  devel¬ 
opment.  Observations  made  during  this  research  effort  concentrated  solely  on 
the  software  development  within  a  weapon  system.  A  sincere  attempt  was 
made  to  isolate  software  development  problems  from  any  other  program  prob¬ 
lems  encountered. 
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G.  ORGANIZATION 

Chapter  II  establishes  the  background  for  the  problem.  Based  on  a  liter¬ 
ature  review,  it  discusses  the  criticality  of  software  development  and  the 
government's  ability  to  effectively  and  efficiently  procure  that  software.  The 
chapter  also  outlines  the  Federal  laws  and  regulations  regarding  software 
development.  Additionally,  it  contains  a  brief  overview  of  Department  of  the 
Navy's  guidance  to  program  managers  and  contracting  officers  concerning  the 
acquisition  of  custom  designed  software. 

Chapter  III  contains  a  description  of  the  contract  types  outline  in  the 
Federal  Acquisition  Regulation  (FAR).  The  program  manager  and  contracting 
officer  must  decide  which  contract  type  best  suits  their  particular  needs.  Any 
of  the  contract  t3q)es  outlined  in  this  chapter  may  be  deemed  suitable  for  the 
procurement  of  custom  designed  software. 

Chapter  IV  provides  an  overview  of  the  two  methodologies  used  in  soft¬ 
ware  development.  The  two  schools  of  thought  can  be  summarized  into  two 
development  philosophies:  Design  to  a  firm  spec,  and  the  use  of  rapid 
prototyping. 

Chapter  V  is  a  collection  of  data  gathered  from  the  research  survey  and 
from  personal  interviews.  It  describes  the  feelings  of  "the  duty  experts  from 
the  field"  with  respect  to  contracting  for  software  development. 

Chapter  VI  draws  conclusions  from  the  data  gathered  and  makes 
recommendations  for  increasing  the  success  rate  of  programs  involving  the 
development  of  mission  critical  software.  The  chapter  concludes  with  recom¬ 
mendations  for  future  research. 
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II.  BACKGROUND 


A.  INTRODUCTION 

There  is  virtually  no  modern  convenience  today  that  does  not  require 
some  piece  of  software  to  be  functional.  From  an  automatic  coffee  maker,  to  a 
microwave  oven,  to  an  automobile,  our  reliance  on  software  has  become  phe¬ 
nomenal.  Likewise,  in  order  to  combat  the  sophisticated  threat,  weapons  sys¬ 
tems  have  become  dependent  on  software  for  their  mission  accomplishment. 

In  February  1979,  the  Comptroller  General  of  the  United  States  (GAO) 
submitted  a  report  to  Congress  titled,  "Contracting  For  Computer  Software 
Development  —  Serious  Problems  Require  Management  Attention  To  Avoid 
Wasting  Additional  Millions"  [Ref.  2].  The  report  cited  contractual  and  man¬ 
agerial  problems  from  the  pre-award  phase  of  the  acquisition  through  con¬ 
tract  execution  that  led  to  cost  overruns,  lengthy  delays  and  failures  to 
deliver  a  usable  product.  The  GAO  investigated  nine  software  developmt  1 1 
programs  in  detail.  Eight  of  the  nine  programs  were  classified  as  "failures 
Only  one  of  the  nine  programs  investigated  produced  software  that  was 
capable  of  being  used  as  delivered.  The  major  contractual  problems  cited  in 
the  report  are  listed  below: 

(1)  Federal  agencies  contract  for  software  development  with  little 
specific  guidance. 

(2)  Agencies  also  overestimate  the  stage  of  system  development  they 
have  reached  before  they  contract. 

(3)  The  lack  of  a  good  contractual  description  of  what  the  contractor  is 
to  do  makes  it  difficult  for  the  agency  to  claim  poor  contractor 
performance. 
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(4)  Agencies  quickly  over  commit  themselves  and  fail  to  control  con¬ 
tractors  through  strict  phasing. 

(5)  Agencies  do  not  manage  software  development  contracts  during 
execution. 

(6)  Agencies  accept  and  pay  for  software  without  adequately  inspecting 
and  testing. 

(7)  Agencies  fail  to  establish  a  single  focal  point  for  communication 
with  contractors. 

(8)  Agencies  do  not  adequately  specify  or  enforce  contract  clauses  for 
recovery  in  the  event  of  poor  performance. 

In  1989,  a  staff  study  entitled,  "Bugs  In  The  Program,  Problems  In  Fed¬ 
eral  Government  Computer  Software  Development  And  Regulation"  [Ref,  3], 
was  submitted  by  the  Subcommittee  on  Investigation  and  Oversight  to  the 
Committee  on  Science,  Space  and  Technology,  U.S.  House  of  Representatives. 
The  study  cited  that. 

Getting  value  for  the  Government’s  money  is  a  recurring  problem. 
Computer  software,  which  is  now  a  major  cost  item  in  many  pro¬ 
curements,  is  not  immune  from  traditional  procurement  problems  such 
as  delay,  cost  overruns  and  poor  performance...  Worse,  the  procure¬ 
ment  system  as  presently  structured  does  not  take  into  account  the 
special  needs  of  computer  software  systems  and  can  compromise 
effective  software  development  from  the  start. 

The  report  further  states  that: 

...software  cannot  be  properly  developed  using  the  welter  of  regulations 
presently  in  force.  While  software  now  drives  system  requirements,  the 
procurement  system  still  focuses  attention  on  hardware... 

Today's  weapon  systems  have  grown  tremendously  more  complex.  Their 
reliance  on  embedded  software  has  increased  in  orders  of  magnitude.  Exam¬ 
ples  of  this  reliance  on  software  are  shown  in  Table  I  [Ref  4],  comparing  an 
early  60's  weapon  system  with  those  of  tomorrow.  Tables  II  and  III  [Ref.  5] 
illustrate  the  cost  trends  and  the  increase  in  the  Department  of  Defense's 
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TABLE  I.  Weapon  Systems  Software  Complexity  Comparison 


WEAPON 

LINES  OF  SOFTWARE  CODE 

F-4 

0  (virtually) 

F-16D 

236,000 

C-17 

750,000 

B-1B 

1.2  million 

ATF 

5  -  7  million 

SDI 

25  million  (estimate) 

*  Lines  of  code  are  often  used  to  describe  the  complexity  of  a  software  program. 
It  should  also  be  noted  that  a  doubling  of  the  lines  of  code  does  not  necessarily 
equate  to  a  doubling  of  complexity,  and  more  likely  results  in  a  program  1 0 
times  rrwre  complex. 

Source:  Kitfield,  James.  "Is  Software  DoD's  Achilles’  Heel?"  Military  Forum 


TABLE  11.  Cost  Trends:  Hardware  Versus  Software 
(percentage  of  total  costs) 


1955 

im 

1979 

1985  (estimate) 

COMPUTER  HARDWARE 

83 

45 

25 

10 

SOFTWARE 

7 

55 

75 

90 

Source;  Glaseman,  Steven.  Comparative  Studies  in  Software  Acquisition 


8 


TABLE  III.  Software  Investment  by  DOD  (billions  of  dollars) 


1979 

1985 

TOTAL  DOD 

SOFTWARE 

7 

12 

EMBEDDED 

SOFTWARE 

5.25 

10+ 

Source:  Glaseman,  Steven.  Comparative  Studies  in  Software  Acquisition 


expenditures  on  software,  respectively,  further  amplifying  this  growing 
reliance  on  software  within  our  major  weapons  systems.  DOD  investment  in 
mission  critical  software  is  predicted  to  be  $30  billion  in  1990  [Ref.  6].  As  this 
dependence  on  software  increased,  industry  has  responded  by  dedicating 
more  of  its  program  resources  to  software  development.  Table  IV  [Ref.  5] 
shows  an  example  of  this  required  increase  by  the  McDonnell  Douglas  Air¬ 
craft  Company. 


TABLE  IV.  McDonnell  Douglas  Resource  Assignment 
(percentage  of  development  personnel) 


12SQ 

I22Q 

12SQ 

(F-4) 

(F-15) 

(F-18) 

NONCOMPUTER  HARDWARE 

98 

74 

57 

EMBEDDED  COMPUTER  HARDWARE 

1 

2 

3 

EMBEDDED  COMPUTER  SOFTWARE 

1 

24 

40 

Source:  Glaseman,  Steven.  Comparative  Studies  in  Software  Acquisition 
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Unfortunately,  the  increased  reliance  on  embedded  software  was  not  met 
with  an  increased  emphasis  on  the  management  of  software  development  on 
the  part  of  both  the  government  and  industry.  Over  the  past  15  years, 
contractors  have  had  little  incentive  for  providing  a  quality  product.  This  does 
not  presume  there  was  an  intent  to  defraud,  but  rather,  that  from  a  manage¬ 
rial  aspect  there  existed  severe  problems  on  both  sides.  The  government 
failed  to  use  its  contractual  "tools/power"  to  identify  standards  of  perfor¬ 
mance,  track  development  programs,  adequately  test  deliverables,  and 
"hammer"  nonperformance.  The  only  thing  that  a  contractor  could  count  on 
seems  to  have  been  the  fact  that  follow-on  contracts  for  software  maintenance 
and  modification  would  somewhat  reward  the  delivery  of  a  non-quality  prod¬ 
uct  [Ref  7].  As  we  have  evolved  into  a  data  processing/software  dependent 
environment,  both  parties  are  now  well  aware  of  the  criticality  of  delivering  a 
quality  product  to  the  end  user.  The  fact  remains,  however,  that  the  man¬ 
agement  of  a  software  development  program  can  be  a  "nightmare"  [Plef  8]. 

Various  reports  at  all  governmental  levels  recognize  that  the  require¬ 
ments  of  today's  rigid  procurement  system  contrast  with  the  flexibility 
afforded  by  the  current  methodologies  used  in  software  development  and 
engineering.  In  other  words,  the  tremendous  flexibility  offered  by  using  soft¬ 
ware  is  often  stifled  by  the  acquisition  system  used  to  procure  it.  With  today's 
shrinking  defense  budget,  the  program  manager  and  contracting  officer  must 
team  up  to  formulate  an  acquisition  strategy  that  will  be  both  workable  and 
effective.  Although  a  change  in  procurement  methodology  regarding  software 
development  is  often  recommended,  in  the  short  term  this  is  unlikely  to  occur. 
Today's  program  managers  and  contracting  officers  must  formulate  their 
procurement  strategies  within  the  existing  laws,  regulations,  and  guidelines 
to  successfully  develop  and  field  the  weapon  systems  of  tomorrow. 
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B.  CURRENT  ENVIRONMENT 

It  has  been  eleven  years  since  that  first  General  Accounting  Office  report 
was  issued  and  we  still  hear  the  same  exact  story.  Time  after  time,  there  are 
system  development,  testing,  production,  and  fielding  delays,  some  of  which 
lead  to  entire  program  cancellations.  These  types  of  results  in  the  develop¬ 
ment  of  custom  software  for  the  Department  of  Defense,  either  for  multipur¬ 
pose  use  or  embedded  within  a  major  weapon  system,  seems  to  be  the  norm 
rather  than  the  exception. 

Recent  reports  indicate  that  software  development  problems  are  a  major 
factor  in  the  current  schedule  delays  and  the  estimated  cost  overruns  as  high 
as  $500  million  on  the  Air  Force’s  C  17  transport  program.  Four  years  after 
the  deployment  of  the  Bl-B,  mission  critical  software  discrepancies  still  exist. 
Eight  years  and  $346  million  were  spent  by  the  Air  Force  in  trying  to  develop 
the  worldwide  military  command-and  -control  information  system  (WIS).  The 
program  was  renamed  and  reassigned  to  the  Defense  Communications 
Agency  after  persistent  software  development  problems.  [Ref  4] 

The  command  and  control  center  for  the  North  American  Aerospace 
Defense  Command  (NORAD)  and  the  U.S.  Space  Command  is  the  Cheyenne 
Mountain  Air  Force  Station.  There,  data  processing  equipment  supporting 
the  command's  tactical  warning  and  attack  assessment  (TW/AA)  mission  was 
scheduled  for  replacement  in  1986.  The  Air  Force  invested  approximately  $72 
million  in  the  initial  software  upgrade  to  the  Semi-automated  Technical  Con¬ 
trol  Unit.  In  December  of  1987,  formal  qualification  testing  of  the  initial  soft¬ 
ware  upgrade  was  completed.  Originally  scheduled  for  a  2-month  period, 
qualification  testing  was  conducted  over  10  months.  The  software  was  consid¬ 
ered  "imstable ",  a  term  applied  to  software  that  is  unpredictable,  and  may  not 
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produce  consistent  results  when  run  against  a  known  set  of  operating 
conditions.  [Ref.  9] 

The  sister  system  to  the  one  described  above  is  the  Air  Force's  Space 
Defense  Operations  Center  (SPADOC)  aiso  at  the  Cheyenne  Mountain  com¬ 
plex.  It  too  has  had  some  major  problems.  After  a  $235  million  investment  in 
the  system  and  a  four  year  schedule  slip,  SPADOC  could  only  meet  seven  of 
the  23  mission  functions  stated  in  the  contract.  Last  year's  estimates  showed 
a  program  cost  growth  of  over  65%  to  $437  milUon  with  the  original  operating 
date  of  June  1988  being  slid  until  sometime  in  fiscal  year  1994.  [Ref.  10] 

Recently,  the  Army  announced  a  26  month  slip  in  IOC  for  their 
Advanced  Field  Artillery  Tactical  Data  System,  an  integral  part  of  their  Tac¬ 
tical  Command  and  Control  System.  The  number  one  cause  for  the  delay, 
cited  in  a  GAO  report,  was  "software  development  and  reliability  problems". 
[Ref  11]  Between  the  program's  approval  in  1986,  and  a  similar  GAO  report 
to  Congress  in  December  1989,  cost  to  complete  estimates  had  risen  by  $600 
million  [Ref.  12].  The  Navy's  next  generation  attack  submarine,  the  SSN-21 
Seawolf,  will  not  be  fully  mission  capable  when  delivered  in  1993.  It's  combat 
system,  the  AN/BSY-2,  will  not  be  fully  operational  because,  "it  will  lack  cer¬ 
tain  necessary  software".  [Ref  13] 

No  Service  is  exempt  from  their  share  of  the  problems.  The  Marine 
Corps  cancelled  the  Marine  Integrated  Fire  and  Air  Support  System  (MIFASS) 
program  after  a  12  year  development  effort  at  a  cost  in  excess  of  $200  million. 
One  of  the  major  reasons  for  cancellation  of  the  program  was  the 
government's  inability  to  manage  the  development  of  the  computer  software 
required  to  meet  the  MIFASS  mission  need  [Ref  14,15]. 

General  Bernard  Randolf,  chief  of  the  Air  Force  Systems  Command, 
summed  up  the  government’s  track  record  in  software  acquisition,  "On 


software  schedules,  we've  got  a  perfect  record:  We  haven't  met  one  yet"  [Ref. 
4;  29]. 


C.  LEGISLATIVE  ACTION 

There  are  two  legislative  actions  that  affect  the  procurement  of  auto¬ 
mated  data  processing  (ADP)  equipment,  to  include  computer  software.  The 
first  was  an  amendment  to  The  Federal  Property  and  Administrative  Services 
Act  of  1949,  and  is  commonly  referred  to  as  the  Brooks  Bill  (PL  89-306). 
Enacted  by  Congress  in  1965,  this  bill  established  the  authority  and  proce¬ 
dure  for  consolidating  all  ADP  activities  such  as  purchasing,  leasing,  equip¬ 
ment  transfer,  servicing,  and  maintenance.  It  made  the  General  Services 
Administration  (GSA)  responsible  for  coordinating  all  ADP  procurement 
activities  in  the  Government.  Federal  agencies  no  longer  had  the  authority  to 
determine  their  own  ADP  policies.  [Ref.  16,17] 

Congress  passed  the  second  legislative  action  regarding  ADP  procure¬ 
ment  in  1981  as  an  amendment  to  the  Fiscal  Year  1982  Appropriations  Act 
(PL  97-86).  Known  as  the  Warner  Amendment,  this  law  excluded  the  De¬ 
partment  of  Defense  from  the  provisions  of  the  Brooks  Bill  when  the  ADP 
equipment: 

...involves  intelligence  activities;  involves  cryptologic  activities  related 
to  national  security;  involves  the  command  and  control  of  military 
forces;  involves  equipment  that  is  an  integral  part  of  a  weapon  or 
weapon  system;  or  subject  to  subsection  (b),  is  critical  to  the  direct 
fulfillment  of  military  or  intelligence  missions. 

Subsection  (b)  specifically  states  that  routine  administrative  and  business 

ADP  equipment  is  not  excluded.  [Ref.  18] 
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D.  ACQUISITION  REGULATIONS 

There  are  two  major  acquisition  regiilations  that  govern  Department  of 
Defense  procurement  efforts.  They  are  the  Federal  Acquisition  Regulation 
(FAR)  and  the  Department  of  Defense  Federal  Acquisition  Regulation  Sup¬ 
plement,  the  DFARS.  While  the  FAR  provides  thousands  of  pages  of 
regulatory  guidance  to  the  contracting  officer  and  the  program  manager,  it 
does  not  differentiate  between  contracting  for  software  development  and 
contracting  for  any  other  goods  or  services.  The  FAR  deals  with  the 
mechanics  of  contracting  vice  strategy  formulation. 

Part  270  of  the  DFARS  addresses  the  acquisition  of  computer  resources. 
The  majority  of  information  provided  in  this  section,  however,  is  merely  a 
reiteration  of  the  legislative  requirements  governing  the  procurement  of 
ADPE.  Other  than  a  one  page  description  of  the  Warner  Amendment  exclu¬ 
sions  to  the  Brooks  Bill,  it  does  not  address  the  procurement  of  custom 
designed  or  embedded  software.  [Ref.  19] 

E.  DEPARTMENT  OF  DEFENSE  PROCUREMENT  GUIDANCE 

The  Department  of  Defense  has  published  various  documents  issued  as 
either  directives  or  instructions  regarding  the  department's  procurement  pro¬ 
grams.  Keeping  in  mind  the  basis  for  this  research  is  managerial  in  nature, 
the  major  acquisition  directives  are  discussed  below.  Many  other  directives 
specifically  addressing  software  development  issues  have  been  promulgated. 
They,  however,  deal  with  specific  technical  requirements  rather  than  overall 
acquisition  strategies  and  will,  therefore,  not  be  addressed. 

1.  DOD  Directive  5000.1 

Department  of  Defense  Directive  5000.1,  dated  1  September  1987, 
and  titled.  Major  and  Non-Major  Defense  Acquisition  Programs,  provides 
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guidance  to  the  Services  regarding  the  classification  of  procurements.  DOD 
procurements  are  classified  as  major  programs  if  their  cost  will  exceed  $1  bil¬ 
lion  in  total  procurement  fimding  or  if  their  total  research,  development,  test 
and  evaluation  (RDT&E)  fimding  will  exceed  $200  million.  There  are  no 
direct  references  to  software  development  programs.  The  two  threshold  crite¬ 
ria,  as  well  as  the  other  provisions  of  the  directive,  would  apply  to  programs 
involving  software  development.  There  are  other  criteria  by  which  a  program 
can  be  designated  as  a  major  acquisition,  such  as  the  amount  of  Congres¬ 
sional  interest.  [Ref.  20] 

2.  DOD  Instruction  5000.2 

"Defense  Acquisition  Program  Procedures,"  DOD  Instruction 
5000.2,  establishes  uniform  procedures  regarding  the  procurement  of  major 
programs.  This  document  mostly  discusses  the  approval  process  through  the 
Defense  Acquisition  Board  (DAB)  and  the  necessary  paperwork  trail  required 
to  document  the  program.  It,  again,  does  not  specifically  address  any  of  the 
aspects  of  software  development  within  the  acquisition  cycle.  [Ref.  21] 

3.  DOD  Directive  5000.3 

The  first  major  publication  that  delineates  guidance  regarding 
software  development  is  DOD  Directive  5000.3,  Test  and  Evaluation  (T&E). 
This  directive  establishes  specific  policy  concerning  the  test  and  evaluation 
of  computer  software.  It  requires  that  the,  "principles  and  methodologies 
provided  in  DOD  5000.3-Manual-3  be  applied  to  the  T&E  of  system  computer 
software."  The  manual  itself  provides  guidance  for  addressing  software  in  the 
Test  and  Evaluation  Master  Plan.  [Ref  22] 

4.  DOD  Directive  5000.29 

DOD  Directive  5000.29,  Management  of  Computer  Resources  in 
Major  Defense  Systems,  "establishes  the  policy  for  the  management  and 
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control  of  computer  resources  (hardware  and  software)  during  the  develop¬ 
ment,  acquisition,  deployment,  and  support  of  major  defense  systems."  This 
directive  requires  programs  to  validate  mission  requirements,  conduct 
appropriate  risk  analyses,  and  provide  for  complete  planning  throughout  the 
computer  resource  life  cycle.  [Ref.  23] 

F.  DEPARTMENT  OF  THE  NAVY  PROCUREMENT  DHIECTIVES 

The  majority  of  the  Department  of  the  Navy’s  procurement  guidance 
implements  the  Federal  and  Department  of  Defense  acquisition  requirements 
within  the  Navy.  Specific  directives  regarding  the  development  of  software 
begins  to  focus  more  on  the  Navy’s  philosophy  for  the  management  of  mission 
critical  computer  resources. 

1.  SECNAVINST  5200.32 

This  instruction,  "Management  of  ECR  in  the  Department  of  the 
Navy,"  implements  DOD  guidance  for  the  management  of  computer  resources 
in  defense  systems  within  the  Department  of  the  Navy.  It  also  promulgates 
additional  policies  and  procedures  for  the  management  of  Navy  weapons, 
communications,  command,  control  and  intelligence  systems  when  embedded. 
[Ref  24] 

2.  SECNAVINST  5200.37 

The  Secretary  of  the  Navy  Instruction  5200.37  entitled,  "Acquisition 
of  Software-Intensive  C2  Information  Systems,"  delineates  acquisition  policy 
for  software-intensive  command  and  control  information  systems.  This 
instruction  promotes,  "routine  user  involvement  in  software  development  pro¬ 
cess...  encourages  rapid  fielding  of  needed  capabilities."  [Ref  25] 
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3.  OPNAVINST  5200.28 


This  instruction,  "Life  Cycle  Management  of  Mission  Critical  Com¬ 
puter  Resources  for  Navy  Systems  Managed  Under  the  Research,  Develop¬ 
ment  and  Acquisition  Process,"  amplifies  SECNAV  and  OPNAV  policies  for 
the  acquisition,  management,  and  life  cycle  support  of  software  and  related 
computer  resources.  Provisions  of  the  Standard  Embedded  Computer 
Resources  Review  Program  are  also  implemented  within  the  Navy.  [Ref.  25] 

4.  MCO  P5000.10C 

Although  Marine  Corps  Order  P5000.10C,  the  "Systems  Acquisition 
Management  Manual,"  states  its  purpose  is  to,  "publish  management  guid¬ 
ance  and  procedures... for  the  acquisition  of  weapon  systems,  computer  re¬ 
sources,  and  equipment..."  it  makes  little  reference  to  software  development. 
The  manual  refers  to  software  development  in  a  single  subparagraph  in  a 
brief  overview  of  Full  Scale  Development  (FSD).  It  does  not  give  the  program 
manager  sufticiciit  guidance  regarding  mission  critical  computer  resources 
(MCCR).  The  order  does  refer  the  reader  to  MCO  5200.23  to  find  the  policy 
and  procedures  for  the  management  of  MCCR.  [Kef  26] 

5.  MCO  5200.23A 

Marine  Corps  Order  5200. 23A,  "Management  of  Mission  Critical 
Computer  Resources  (MCCR)  in  the  Marine  Corps,"  implements  the  policies 
and  guidance  promulgated  by  the  Department  of  Defense  and  the  Secretary 
of  the  Navy.  It  establishes  a  Marine  Corps  policy  for  the  acquisition  and 
management  of  MCCR.  The  order  states  its  purpose  as,  "to  reduce  the  life 
cycle  support  costs  of  all  Marine  Corps  mission  critical  systems  by  reducing 
the  proliferation  of  mission  critical  system  hardware  and  software."  [Ref.  27] 
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6.  Naval  Systems  Command  Instructions 

Various  instructions  and  guidance  have  been  promulgated  by  each 
of  the  Navy's  Systems  Commands.  The  Naval  Air  Systems  Command 
(NAVAIR),  the  Naval  Sea  Systems  Command  (NAVSEA),  and  the  Space  and 
Naval  Warfare  Systems  Command  (SPAWAR)  have  all  issued  their  own  pro¬ 
gram  manager's  policy  guidance  regarding  software  development.  Most  of 
these  instructions  direct  the  implementation  of  DOD  guidance  and  assign 
areas  of  responsibility  within  the  command.  [Ref.  25] 

7.  Tactical  Digital  Standards  (TADSTAND) 

A  series  of  policy  documents  have  been  developed  by  the  Navy 
regarding  system  software  development  and  support.  Published  as  Tactical 
Digital  Standards  (TADSTAND's),  they  provide  guidance  for  the  standardiza¬ 
tion  of  such  things  as;  embedded  computers,  computer  peripherals, 
input/output  interfaces,  standardized  languages,  software  testing  and  docu¬ 
mentation  requirements  for  the  Navy's  mission  critical  systems. 
TADSTAND's  serve  to  both  mandate  the  use  of  certain  DOD  Standards  in 
software  development  and  testing,  and  also  identify  the  Navy's  unique  mis¬ 
sion  critical  computer  resources  requirements.  [Ref  8,25] 

G.  SPECIFICATIONS  AND  STANDARDS 

The  Department  of  Defense  publishes  a  series  of  reference  documents 
which  can  be  used  by  all  parties  in  military  procurement.  These  docu¬ 
ments  fall  into  three  categories;  Military  Handbooks,  Military  Specifications 
and  Military  Standards.  Handbooks  bring  together  procedural  and  technical 
or  design  information  related  to  components,  processes  or  services. 
Specifications  are  complete  and  detailed  descriptions  of  products  which  are 
either  strictly  military  in  nature,  or  are  modified  commercial  products 


requiring  special  features  to  satisfy  military  mission  needs.  Standards 
describe  engineering  and  management  processes,  methods  and  design 
criteria,  testing  techniques,  and  data  collection  requirements.  It  is  important 
to  remember  that  handbooks,  specifications,  and  standards  are  merely  guides 
to  assist  the  government's  program  manager  and  contracting  officer.  They  are 
not  laws.  Once  placed  in  a  contract,  however,  they  become  legally  binding. 
The  major  specifications  and  standards  regarding  software  development  are 
outlined  in  this  section. 

1.  DOD.STD-1467 

Although  primarily  written  as  an  Army  document,  "Military  Stan¬ 
dard  Software  Support  Environment,"  is  available  for  use  by  all  Departments 
of  DOD.  This  document  provides  the  uniform  minimum  requirements  for  the 
contractor  to  define  a  Development  Software  Support  Environment  (DSSE), 
to  ensure  compatibility  with  the  government’s  designated  Life  Cycle  Software 
Support  Environment  (LCSSE).  In  other  words,  the  program  manager,  the 
contracting  officer,  and  the  contractor  must  plan  for  the  life  cycle  support  of 
deliverable  software  throughout  the  development  cycle.  This  makes  all  three 
parties  not  only  aware,  but  also,  responsible  for  software  support  after  the 
software  becomes  operational.  The  standard  is  written  in  such  a  way  as  to  not 
dictate  a  specific  approach,  but  to  allow  the  software  contractor,  "the  flexibil¬ 
ity  to  develop  software  and  manage  the  contract  in  accordance  with  the  con¬ 
tractor's  best  judgment  and  practices."  [Kef.  28] 

2.  DOD.STD.1703 

This  standard,  "Software  Product  Standards,"  looks  at  the  software 
development  process  more  from  a  manager's  perspective.  It  is  the  best  docu¬ 
ment  to  use  as  a  guide  in  the  formulation  of  an  acquisition  strategy  for  a 
software  development  program.  It  provides  the  program/project  manager 
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valuable  guidance,  without  giving  the  impression  that  the  recommendations 
provided  are  mandated.  DOD-STD-1703  encourages  the  manager  to  tailor 
program  requirements,  up-front,  placing  emphasis  on  the  requirement  for 
proper  dociunentation  of  the  contractor's  efforts,  rather  than  attempting  to 
control  or  direct  those  efforts.  Table  V  provides  some  basic  questions  that  the 
program  manager  must  answer  when  tailoring  program  requirements. 


TABLE  V.  Factors  that  Affect  Program  Requirements 


1 .  How  large  and  how  complex  is  the  software  product  ? 

2.  Who  will  use  the  software  product  ? 

3  How  long  will  it  be  used  ? 

4.  Will  someone  be  required  to  provide  life-cycle  support  for  the  product  after 
it  is  developed  ? 

5.  During  development,  which  of  the  following  does  the  customer  consider 
most  important;  cost  control,  schedule  control,  or  functional  capability  ? 

6.  What  are  the  development  risks  ? 

7.  How  large  is  the  development  staff  ? 

Source:  DOD-STD-1703 


Table  VI  provides,  "the  axioms  of  software  development,"  some  very  basic  and 
simple  guidance,  but  at  the  same  time  some  very  valid  and  valuable  rules  to 
live  by.  [Ref.  29] 
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TABLE  VI.  Axioms  of  Software  Development 


1 .  To  be  successful,  a  software  project  must  be  well-managed. 

2.  Good  managennent  requires  planning,  attention  to  detail,  and  discipline. 

3.  Documentation  is  a  primary  vehicle  with  which  managers  manage. 

4.  Until  a  software  product  is  tested,  no  one  will  know  its  quality. 

5.  Both  customers  and  users  have  a  vital  role  to  play  in  any  software 
development  effort;  documentation  gives  them  the  information  necessary 
to  perform  their  role. 

Source:  DOD-STD-1703 


3.  DOD-STD-2167A 

Published  in  1988,  the  standard  for  "Defense  Systems  Software 
Development,"  is  the  latest  guidance  promulgated  by  the  DOD.  It  establishes 
uniform  standards  for  software  development  that  are  applicable  throughout 
the  system's  life  cycle.  DOD-STD-2167A  establishes  the  requirements,  how¬ 
ever,  it  does  not  specify  how  the  contractor  is  to  meet  those  requirements.  For 
example,  the  requirement  to  have  a  software  management  structure  in  place 
is  worded  as  follows;  "...the  contractor  shall  implement  a  process  for 
managing  the  development  of  the  deliverable  software."  Again,  emphasis  is 
placed  on  the  program  manager's  ability  to  determine  his  particular  needs, 
and  tailor  his  project  requirements  accordingly.  All  aspects  of  software 
design,  development  and  testing  are  addressed  in  this  standard.  [Ref.  30] 


4.  DOD-STD-2168 

DOD  Standard  2168,  "Defense  System  Software  Quality  Program," 
contains  the  requirements  for  the  development,  documentation  and  imple¬ 
mentation  of  a  software  quality  program.  It  interprets  the  applicable  quality 
requirements  set  forth  in  MIL-Q-9858,  the  most  stringent  of  the  quality 
assurance  specifications,  for  software.  Published  in  1988,  it  supersedes  the 
long  standing  software  quality  assurance  gmdance  found  in  MIL-S-52799A. 
[Ref.  31] 

5.  MIL-HDBK-286 

This  handbook,  "Software  Quality  Evaluation,"  is  currently  avail¬ 
able  only  as  a  draft,  and  is  intended  to  be  used  with  DOD-STD-2168.  It  pro¬ 
vides  guidelines  for  applying  the  requirements  of  the  Department's  Software 
Quality  Program.  [Ref.  32] 

6.  MIL-HDBK.287 

Military  Handbook-287,  published  in  1989,  is  "A  Tailoring  Guide  for 
DOD-STD-2167A,  Defense  System  Software  Development.”  It  provides  just 
that,  guidance  to  the  program  managers  and  other  program  office  personnel 
for  tailoring  the  requirements  of  2167  for  a  software  development  or  support 
contract.  [Ref.  33] 

7.  MIL.HDBK-782 

This  handbook,  "Software  Support  Environment  Acquisition,"  was 
written  to  provide  personnel  not  familiar  with  the  requirements  of  DOD-STD- 
1467  the  fundamentals  required  to  ensure  supportability  on  contracted  soft¬ 
ware  development  programs.  Emphasis  is  placed  on  the  concern  for  software 
supportability  beyond  the  contractual  delivery  date  and  its  subsequent  oper¬ 
ational  use.  It  is  intended  for  use  by  all  personnel  in  the  procurement  process 
responsible  for  ensuring  life  cycle  software  supportability.  [Ref.  34] 
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H.  MANAGEMENT  GUIDANCE 

There  are  many  documents  that  are  readily  available  to  assist  the 
Navy/Marine  Corps  program  manager  and  his  staff  through  the  various 
phases  of  an  acquisition.  Although  these  documents  are  informal,  that  is, 
they  are  not  published  through  the  Navy  Directives  Issuance  System,  they 
often  provide  the  insight  necessary  for  the  program  manager  to  perform  suc¬ 
cessfully.  Three  examples  that  would  be  pertinent  to  the  acquisition  of  com¬ 
puter  software  are  cited  below. 

1.  RDT&E/Acquisition  Management  Guide 

The  Navy's,  "Research  Development  Test  &  Evaluation/Acquisition 
Management  Guide  -  11th  Edition,"  published  in  January  of  1989  misses  the 
mark  when  it  comes  to  assisting  the  program  manager  with  software  devel¬ 
opment  issues.  There  is  a  only  one  paragraph  addressing  mission  critical 
computer  resources.  Unfortunately,  it  does  not  address  development  or  test¬ 
ing  requirements,  but  rather,  the  requirement  for  an  approved  Computer 
Resources  Life  Cycle  Management  Plan  (CRLCMP)  prior  to  the  Milestone  II 
decision  point.  It  says  in  part: 

Advanced,  fully  integrated  weapons,  avionics,  intelligence,  and 
command,  control  and  communications  technologies  are  gaining 
increasing  importance  in  Navy  and  inter-service  weapons  systems.  The 
nuclei  of  such  integrated  systems  are  embedded  Mission  Critical 
Computer  Resources  (MCCR)...[Ref.  35:  2-19] 

Although  the  document  seemingly  recognizes  the  "increased  importance"  of 

embedded  software,  it  fails  to  provide  any  further  guidance.  [Ref  35] 

2.  Acquisition  Strategy  Guide 

In  1984,  the  Defense  Systems  Management  College,  Fort  Belvoir, 
Vi’-ginia  published  the  "Acquisition  Strategy  Guide."  Its  purpose  was  to  pro¬ 
vide,  "information  that  Program  Managers  should  find  useful  in  structuring. 
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developing,  and  executing  an  acquisition  strategy."  Like  the  RDT&E  manual, 
this  publication  addresses  the  mechanical  aspects  of  strategy  formulation.  It 
fails  to  identify  any  unique  strategy  formulation  reqmrements  when  dealing 
with  the  development  of  custom  designed,  embedded  software.  It  does  not 
recognize  the  difference  between  hardware  and  software;  it  merely  addresses 
programs  as  a  whole.  [Ref.  36] 

3.  Navy  Program  Manager's  Guide 

The  "Navy  Program  Manager’s  Guide"  is  the  first  publication 
found  that  places  the  importance  of  software  development  management  in 
perspective: 

The  combat  launch  of  an  aircraft  without  its  missiles  or  the 
sortie  of  a  submarine  without  its  torpedoes  makes  successful  destruc¬ 
tion  of  enemy  forces  highly  unlikely.  To  any  professional,  this  fact  is 
obvious,...  Far  fewer,  however,  know  that  a  weapon  systems  with  full 
armament  installed  may  be  equally  failure-prone  because  of  embedded 
computer  resource  (ECR)  deficiencies.  ...PM's  universally  have  become 
aware  that  we  can  no  longer  fly,  dive,  steam  or  fight  without  ECRs  in 
most  of  our  weapons.  We  use  ECRs  to  make  our  systems  operate,  to 
test  them,  to  produce  them,  to  adapt  them,  and  to  keep  them  respon¬ 
sive  to  changing  threats.  [Ref.  8:  4-67] 

The  manager  is  made  aware  of  the  critical  role  that  embedded  computer 
resources,  and  their  management,  play  in  mission  accomplishment.  The  guide 
provides  the  program  manager  a  "heads-up"  stating,  "Software  management 
has  been  recognized  by  successful  PM's  as  requiring  early,  intensive,  continu¬ 
ing  management."  It  goes  on  to  provide  excellent  guidance  in  general  terms 
and  recommends  various  sources  of  assistance  available  within  the  Navy. 
[Ref  8] 

I.  SUMMARY 

This  chapter  has  provided  the  background  for  the  complex  environment 
in  which  program  managers  and  contracting  officers  must  operate  when 
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procuring  embedded  software.  The  software  development  costs  associated 
with  major  weapon  system  programs  can  be  astronomical.  Armed  with  very 
little  formal  guidance  these  "procurement  experts"  must  operate  within  the 
law  to  ensure  a  quality  product  is  delivered  on  time  and  on  budget.  When  it 
comes  to  contracting  for  custom  software  the  program  manager  and  contract¬ 
ing  officer  are  actually  afforded  alot  of  flexibility.  They  must  rely  heavily  on 
their  common  sense  approach  and  good  business  judgment.  Problem  pro¬ 
grams  quickly  come  under  the  scrutiny  of  the  Congress,  the  GAO,  the  IG's, 
and  various  auditors.  Chapter  III  will  discuss  the  specific  types  of  contracts 
available  to  the  program  manager  and  contracting  officer  for  the  procurement 
of  embedded  software. 
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III.  CONTRACTING  FOR  SOFTWARE  DEVELOPMENT 
A  INTRODUCTION 

The  single  most  important  document  in  the  procurement  of  custom 
designed  software  is  the  contract  itself  The  contract  acts  as  the  legal  basis  for 
performance,  and  the  importance  of  stating  in  the  contract  what,  in  fact,  is 
intended  can  not  be  overemphasized.  The  government  and  the  contractor 
must  collectively  enter  into  this  agreement,  understanding  that  it  is  a  mutual 
effort  with  both  parties  having  their  own  responsibilities.  Various  types  of 
contracts  are  outlined  in  the  Federal  Acquisition  Regulation  (FAR).  The  first 
consideration  in  choosing  a  contract  type  is  usually  that  of  risk  assumption. 
The  spectrum  of  contract  types  range  from  a  Ccst-Plus-Fixed-Fee  type  con¬ 
tract  in  which  the  government  assumes  the  majority  of  the  risk,  to  a  Firm- 
Fixed-Price  type  contract  which  places  the  greatest  risk  on  the  contractor. 
This  chapter  will  provide  a  brief  overview  of  the  types  of  contracts  that  may 
be  used  for  the  acquisition  of  custom  developed  software. 

B.  COST  REIMBURSABLE  CONTRACTS 

A  cost  reimbursable  contract  is  one  that  allows  the  contractor  to  receive 
payment  for  all  costs  that  are  deemed  "allowable  and  allocable".  These  type 
contracts  do  not  contain  a  cost  ceiling,  thus  the  government  assumes  the 
majority  of  the  risk.  The  contractor  provides  the  government  with  an  esti¬ 
mate  to  complete  the  task,  however,  the  contractor  is  not  bound  by  that  esti¬ 
mate.  It  is  possible  to  expend  all  of  the  funds  allocated  for  a  particular 
program  and  have  a  situation  where  the  contractor  is  not  legally  bound  to 
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produce  a  deliverable.  The  following  are  variations  of  this  cost  reimbursable 
concept. 

1.  Cost-Plus-Fixed-Fee 

A  cost-plus-fixed-fee  (CPFF)  contract  is  a  cost  reimbursement  con¬ 
tract  that  establishes  a  fixed  fee  payment  to  the  contractor.  The  contract  price 
is  based  on  a  negotiated  cost  to  complete  the  required  task  plus  a  negotiated 
contractor's  fee.  The  contractor  is  not  legally  bound  by  this  negotiated  cost 
figure.  As  implied  by  the  contract  name,  the  contractor's  fee  remains  fixed 
regardless  of  total  contract  price  to  the  government.  The  contractor  receives 
the  negotiated  fee  regardless  of  the  actual  costs  incurred.  At  the  same  time, 
the  contractor  has  a  legal  claim  for  his  actual  costs.  Under  this  arrangement 
the  government  assumes  the  majority  of  the  risk,  and  the  contractor  has  little 
incentive  to  control  costs.  [Ref  37:  16.306] 

2.  Cost-Plus-Incentive-Fee 

In  a  oost-plus-incentive-fee  (CPIF)  contract  the  government  and  the 
contractor  agree  to  a  target  cost  based  on  the  required  task.  Like  the  CPFF 
contract,  the  contractor  has  the  right  to  be  reimbursed  for  all  allowable 
expenses  incurred.  The  difference,  however,  is  that  the  contractor's  fee  is 
adjusted  by  using  a  formula  that  compares  the  total  allowable  costs  to  the 
negotiated  target  cost.  Thus,  the  contractor  has  the  incentive  to  effectively 
manage  cost  control.  At  the  onset  of  the  effort,  a  target  cost,  target  fee,  a  min¬ 
imum  and  maximum  fee,  and  an  adjustment  formula  are  negotiated.  If  the 
contractor  completes  the  task  below  the  target  cost  the  fee  is  adjusted  upward 
based  on  a  cost  share  ratio  established  in  the  contract.  Likewise,  if  the  con¬ 
tractor's  actual  costs  are  over  the  target  cost  the  fee  will  be  adjusted  down¬ 
ward.  The  minimum  and  maximum  fee  figures  are  included  in  the  contract, 
on  the  one  hand,  to  protect  the  contractor  from  being  in  the  position  of  having 
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to  work  for  no  fee,  and  on  the  other,  to  prevent  the  contractor  from  taking  cost 
short  cuts  to  merely  increase  his  profit.  This  type  contract  is  best  suited  for  a 
development  effort  when  a  realistic  target  fee  and  target  cost  can  be  negoti¬ 
ated  that  will  effectively  incentivize  the  contractor  to  control  costs.  [Ref.  37: 
16.404-1] 

3.  Cost-Plus-Award-Fee 

A  cost-plus-award-fee  (CPAP)  contract  is  also  a  reimbursable  type 
contract.  Under  this  type  arrangement  the  contractor's  fee  is  divided  into  two 
parts.  The  first  consists  of  a  base  fee  established  at  the  onset  of  the  contract, 
and  the  second,  consists  of  an  award  amount  or  pool  that  is  set  aside.  The 
contractor  may  earn  all  or  part  of  this  award  fee  pool  based  on  the  govern¬ 
ment's  "judgmental  evaluation"  of  the  contractor's  performance.  The  intent  is 
to  monetarily  reward  the  contractor  for  performance  above  the  minimally 
accepted  standard  set  at  the  beginning  of  the  contract.  Like  other  incentive 
type  contracts,  the  contractor  can  be  rewarded  for  technical  performance,  sup- 
portability  and  reliability  achievements,  or  superior  quality.  A  service  con¬ 
tract  to  run  a  military  dining  facility  can  be  used  as  a  simple  example.  An 
award  pool  could  be  structured  to  incentivize  the  quality  of  the  contractor's 
food,  service,  and  cleanliness  requirements  beyond  that  which  was  estab¬ 
lished  as  the  minimum  acceptable  standard  in  the  contract.  A  government 
appointed  person  or  committee  periodically  awards  portions  of  the  monetary 
pool  based  on  their  subjective  impressions  of  the  contractor's  performance. 
[Ref  37:  16.404-2] 

C.  FIXED  PRICE  CONTRACTS 

In  a  fixed  price  type  contract  the  contractor  is  legally  bound  to  deliver 
the  goods  or  services  for  a  predetermined  price  regardless  of  his  actual  costs. 


This  t3T)e  contract  is  most  appropriate  in  a  production  type  buy  where 
product  designs  have  been  stabilized.  "Key  features  of  a  fixed-price  contract 
are:  contractor  promises  to  deliver  on  time,  per  specification,  for  a  fixed  price, 
and  the  government  promises  to  pay  the  fixed  price  if  the  product/service  con¬ 
forms  to  the  contract"  [Ref.  38].  In  this  type  agreement,  the  contractor 
assumes  majority  of  the  cost  risk.  There  several  variations  of  the  fixed  price 
type  contract  listed  in  Part  16  of  the  FAR.  They  are  listed  below: 

•  Firm-Fixed-Price  (FFP) 

•  Fixed-Price  with  Economic  Price  Adjustment  (FPE) 

•  Fixed-Price  Incentive,  Firm  Target  (FPIF) 

•  Fixed-Price  Incentive,  Successive  Targets  (FPIS) 

•  Fixed-Price  with  Prospective  Price  Redetermination  (FPRP) 

•  Fixed-Price  Retroactive  Price  Redetermination  (FPRR) 

•  Firm-Fixed-Price,  Level  of  Effort  (FFP,  LOE) 

When  contracting  for  software  development,  typically  used  contract  types  can 
be  placed  in  four  general  categories.  They  are  discussed  below. 

1.  Firm-Fixed-Price 

The  simplest  of  any  contracts  is  the  firm-fixed-price  agreement.  In 
this  agreement,  the  contractor  is  paid  the  contract  price  upon  acceptance  of 
the  work.  There  are  no  adjustments  to  total  price  regardless  of  the  actual  cost 
to  perform.  The  contractor's  profit  or  loss  posture  has  no  bearing  on  the  price 
paid  once  the  contract  is  awarded.  This  arrangement,  "provides  maximum  in¬ 
centive  for  the  contractor  to  control  costs  and  perform  effectively  and  impose 
a  minimum  administrative  burden  upon  the  contracting  parties"  [Ref.  37: 
16.202-1]. 
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2.  Fixed-Price-Incentive 

Fixed-price-incentivc  contracts  have  one  of  two  options.  They  will 
either  have  an  established  Firm  Target  (FPIF)  or  a  series  of  Successive  Tar¬ 
gets  (FPIS).  Both  of  these  arrangements  provide  for  an  adjustment  to  the 
contractor's  profit  based  on  the  relationship  between  the  final  negotiated  total 
contract  cost  and  the  target  cost  established  at  contract  award.  At  the  onset 
of  the  contract,  a  target  cost,  target  profit,  ceiling  price,  and  a  share  ratio  for 
establishing  the  final  profit  and  price  are  negotiated.  At  the  completion  of  the 
contract  a  final  contract  price  is  determined,  and  the  contractor's  profit 
posture  reflects  the  application  of  this  negotiated  share  formula.  When  the 
final  cost  is  more  than  the  target  cost,  the  contractor's  profit  will  be  less  than 
the  target  profit,  and  in  some  cases  may  even  result  in  a  net  loss.  With  profit 
tied  to  this  share  ratio,  the  contractor  is  incentivized  to  control  costs.  The  ceil¬ 
ing  is  established  to  protect  the  government  from  paying  more  than  that  total 
price  for  the  contract.  Unlike  the  cost  reimbursable  contracts,  the  contractor 
must  satisfactorily  complete  the  contract  task  in  the  form  of  a  service  or 
deliverable  regardless  of  coot.  The  o.iU  uifTcicxiCe  between  the  two  FPI 
contracts  is  the  time  at  which  the  share  formula  is  applied.  In  a  successive 
target  situation,  the  share  ratio  is  computed  at  different  times  throughout  a 
production  effort.  The  FPIS  contract  would  be  appropriate  in  a  situation 
where  the  cost  or  pricing  data  were  not  adequate  at  the  beginning  of  the  pro¬ 
duction  run  to  negotiate  a  firm-fixed-price  contract.  [Ref  37:  16.403] 

3.  Fixed-Price  with  Economic  Adjustment 

When  the  contracting  officer  determines  that  the  market  conditions 
are  so  unstable  that  the  either  government  or  the  contractor  need  to  be 
afforded  some  level  of  protection,  an  economic  adjustment  type  contract  is 
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appropriate.  Section  16.203-1  of  the  FAR  identifies  three  contingencies  for 
price  adjustments: 

(1)  Adjustments  based  on  established  prices. 

(2)  Adjustments  based  on  actual  costs  of  labor  or  material. 

(3)  Adjustments  based  on  cost  indexes  of  labor  or  material. 

The  upward  or  downward  adjustments  may  be  up  to  ten  percent  of  the  con¬ 
tract's  base  price.  The  price  adjustment  clause  in  the  contract  lists  specific 
contingencies  upon  which  the  price  adjustment  will  take  effect.  Other  than 
these  delineated  events  the  contract  is  executed  virtually  the  same  as  a  firm- 
fixed-price  contract.  This  type  of  contract  would  not  normally  be  used  in  a 
software  development  effort,  it  is  however  possible,  and  is  included  for  pur¬ 
poses  of  this  discussion.  [Ref  37:  16.203] 

4.  Level  of  Effort 

Typically  used  for  a  feasibility  type  study  in  a  research  or  develop¬ 
ment  effort,  the  contract  establishes  a  firm- fixed -price  for  a  specified  level  of 
effort  rather  than  an  end  product.  Payment  is  based  on  the  level  of  work 
expended  and  not  the  achieved  results.  The  scope,  of  the  work  is  loosely 
defined  allowing  the  contractor  the  greatest  flexibility.  For  example,  the  gov¬ 
ernment  might  hire  a  contractor  to  perform  1000  man-hours  of  research.  The 
contractor  assumes  almost  no  financial  risk,  in  that,  he  will  get  paid  for 
expending  those  1000  hours  regardless  of  the  outcome.  [Ref.  37:  16.207] 

D.  OTHER  CONTRACT  OPTIONS 

The  FAR  also  states  that  contracts,  "may  be  of  any  type  or  combination 
of  types  that  will  promote  the  Government’s  interest..."  [Ref.  37  16. 102. b] 
Gerald  Lee  Francom,  while  a  student  at  the  Naval  Postgraduate  School, 
explored  the  feasibility  of  using  a  Fixed-Price-Award-Fee  contract.  He 
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combined  the  advantages  of  the  Fixed-Price  contract  with  the  incentive 
provisions  of  an  Award  Fee  contract  and  proposed  that  this  type  contract  be 
recognized  within  the  FAR.  [Ref.  39] 

The  Air  Force  has  in  the  past  used  this  combination  of  contract  types 
with  great  success.  The  improvement  program  for  the  B-IB  bomber  Computer 
Integrated  Test  System  (CITS)  was  awarded  under  a  Fixed  Price  Incentive 
Firm  with  an  Award  Fee  (FPIF/AF).  The  original  contract  specification  called 
for  a  two  percent  or  less  false-alarm  rate  which  was  met  by  the  contractor. 
After  the  system  was  delivered  this  government  specified  rate  was  deemed  to 
be  inadequate  by  the  end  users.  Thus,  a  program  to  reduce  the  number  of 
CITS  false-alarms  was  undertaken.  Under  the  FPIF/AF  pricing  arrangement, 
the  contractor  was  incentivized  to  provide  high  management  emphasis  on 
quality,  timeliness,  and  cost-effectiveness  during  this  effort.  The  resultant 
program  produced  a  false-alarm  rate  that  was  below  .03  percent.  In  this  case, 
the  incentive  contract,  "resulted  in  a  true  win/win  situation  for  both  the  gov¬ 
ernment  and  the  contractor."  [Ref.  40] 

E.  CHOOSING  THE  PROPER  CONTRACT 

"Selecting  the  contract  type  is  generally  a  matter  for  negotiation  and 
requires  the  exercise  of  sound  judgment"  [Ref.  37:  16.103].  Contracting  offi¬ 
cers  are  given  the  latitude,  in  most  cases,  to  choose  the  type  of  contract  that 
best  suits  their  particular  needs.  As  previously  stated,  the  gaiting  factor  in 
this  decision  is  usually  the  question  of  "risk  assumption".  The  two  risks  that 
are  analyzed  prior  to  choosing  a  contract  type  are  cost  risk  and  technical  risk. 
A  third  risk  in  the  formula  is  schedule,  however,  schedule  risk  is  mostly  a 
function  of  the  other  two.  Dr.  Harvey  J.  Gordon,  in  an  article  discussing  the 
role  of  the  contract  stated: 
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Not  all  major  weapon  system  contracts  are  designed  to  accomplish 
precisely  the  same  purpose.  There  are,  however,  invariants  that  thread 
a  common  course.  Despite  differences  in  contracting  technique,  we 
strive  always  for  the  lowest  possible  cost,  timely  delivery,  and  maxi¬ 
mum  technical  accomplishment  within  cost  and  schedule  constraints. 
[Ref.  41:  30] 

Every  program  manager  and  contracting  officer  attempts  to  accomplish  this 
not  so  easy  task.  An  accurate  assessment  and  subsequent  assignment  or 
assumption  of  risk  is  a  critical  factor  in  achieving  this  goal.  Selection  of  the 
proper  type  of  contract  assigns  cost,  schedule  and  technical  risk  to  the  appro¬ 
priate  party. 


F.  SUMMARY 

This  chapter  provided  an  overview  of  the  specific  contract  types  outlined 
in  the  Federal  Acquisition  Regulation.  It  also  showed  that  the  contracting 
officer  is  given  the  flexibility  to  choose  the  contract  type  that  bests  suits  the 
program's  needs.  The  ultimate  goal  is  to  achieve  a  win/win  situation  using  the 
appropriate  contract  type.  The  government  will  receive  a  quality  produce/ 
service  and  the  contractor  will  earn  a  fair  and  reasonable  profit. 
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IV.  SOFTWARE  DEVELOPMENT 


A.  SOFTWARE  DEVELOPMENT  PHILOSOPHY 

The  classical  approach  to  software  development,  known  as  the  "waterfall 
model,"  is  illustrated  in  Figure  4.1  [Ref.  42].  This  approach  is  quite  compati¬ 
ble  with  the  government’s  current  contracting  philosophy.  Using  this  classical 
software  development  method,  requirements  are  defined  "up-front",  similar 
to  the  acquisition 


Figure  4-1.  Classic  Software  Life  Cycle 

process  of  publishing  a  detailed  specification  or  statement  of  work.  A  contract 
is  awarded  delineating  the  specific  design  criteria  in  detail.  The  contractor 
designs  and  tests  in  accordance  with  the  contractual  specifications.  Very  little 
flexibility  is  afforded  to  either  the  contractor  or  the  government  once  these 
requirements  are  integrated  into  the  development  contract.  System  develop¬ 
ment,  especially  in  the  case  of  complex  weapon  systems,  very  rarely  follows 
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the  clearly  defined  sequential  waterfall  model.  In  the  real  world,  users  do  not 
easily  communicate  or  sometimes  even  know  their  true  requirements.  Con¬ 
tracting  officers  do  not  easily  translate  user  requirements  into  definitized 
contracts.  Software  engineers  do,  in  fact,  need  the  flexibility  during  design  to 
accommodate  the  inherent  unknowns  in  software  development. 

The  more  modern  approach  to  development,  illustrated  in  Figure  4.2 
[Ref.  42],  uses  what  is  known  as  "rapid  prototyping."  Using  this  concept,  the 
user  will  define  a  group  of  objectives  for  the  software  to  be  built.  This  quick 


Figure  4-2.  Prototyping 


design  concentrates  on  generalized  functions  of  the  software,  paying  particu¬ 
lar  attention  to  those  aspects  of  the  program  which  are  visible  to  the  user. 
The  quick  design  evolves  into  a  prototype.  The  prototype  then  serves  as  the 
tool  to  identify  the  user’s  true  requirements.  It  is  probably  safe  to  assume 
that  in  the  majority  of  design  efforts  the  user  does  not  have  a  real  handle  on 
what  is  required,  at  least  to  the  level  of  detail  that  would  be  necessary  to 
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begin  full  scale  software  development.  Initial  concepts  will  most  likely 
change.  The  impact  of  this  change  will  vary  depending  on  the  stage  of  the 
program.  Dr.  Roger  S.  Pressman's  comparison  of  the  cost  of  change  is  illus¬ 
trated  in  Figure  4.3  [Ref.  42:  17].  The  use  of  an  iterative  process  through  pro¬ 
totyping  allows  maximum  interface  between  the  ultimate  user  and  the 
developer.  Prototyping  prior  to  full  development  accommodates  this 
inevitable  change.  It  does  so  at  the  easiest  and  least  costly  point  in  the  soft¬ 
ware  life  cycle.  Thus,  the  user  better  identifies  his  requirements  and  the 
developer  better  understands  those  requirements. 


COST TO 
CHANGE 


DEFINITION 


DEYELOPIvENT 


MAINTENANCE 


Figure  4.3.  Cost  impact  of  Change 


The  program  manager's  ability  to  create  this  environment  where  the  end 
user  plays  an  integral  role  in  the  early  stages  of  development  is  one  of  the 
essential  elements  in  a  successful  program.  Afterall,  who  better  knows  the 
true  mission  requirements  than  the  end  user.  On  the  other  hand,  extreme 


care  must  be  taken  to  not  allow  the  user  to  "gold  plate"  the  requirements,  just 
because  the  software  capability  exists.  If  the  requirement  is  to  track  ten  in¬ 
bound  airborne  targets  simultaneously,  then  that  should  be  the  design  crite¬ 
ria.  The  attitude  that  it  "sure  would  be  nice"  to  track  100  inbound  targets 
should  be  stifled  if  the  tactical  mission  and/or  the  hardware  will  not  support 
that  requirement.  The  fact  that  the  software  can  "easily"  accomplish  this  task 
is  not  relevant.  Respondents  to  this  thesis'  survey  overwhelmingly  cited  the 
inability  to  identify  the  user's  requirements  as  the  number  one  problem  in 
software  development  today.  Here,  the  program  manager  and  contracting 
officer  assume  the  role  of  mediator. 

The  prototyping  philosophy  seems  to  be  accepted  by  industry  (and,  as  a 
matter  of  fact,  the  government)  as  the  best  approach  to  a  new  software  devel¬ 
opment  program.  Unfortunately,  there  exists  an  inherent  conflict  between 
this  approach  and  the  today's  acquisition  system.  The  government's  procure¬ 
ment  system  operates  on  the  premise  that  all  work  will  be  specified  in  detail 
before  the  contract  is  awarded.  The  prototype  approach  uses  broad  fimctional 
descriptions  to  loosely  define  the  end  product's  objectives,  not  the  work  to  be 
performed,  ergo,  the  conflict.  The  program  manger  and  contracting  officer's 
ultimate  challenge  is  to  choose  the  best  design  philosophy  and  fit  it  into  a 
contractual  mechanism. 

B.  SOFTWARE  DEVELOPMENT  ENVIRONMENT 

As  program  managers  and  contracting  officers,  on  both  sides,  become 
more  software  literate,  the  ability  to  contract  for  software  development 
should  become  somewhat  easier.  At  least  to  the  extent  that  all  players  in  the 
contracting  process  will  have  some  understanding  and  appreciation  for  the 
differences  between  hardware  and  software  development.  In  addition  to  our 
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inability  to  identify  user  requirements,  a  second  problem  area  is  simply  our 
unfamiliarity  with  the  software  development  process.  Typically,  today's  pro¬ 
gram  manager  was  brought  up  though  the  ranks  in  one  of  the  technical 
disciplines  other  than  software  engineering.  Just  ask  any  of  them  a  hardware 
related  question  at  a  program  review,  and  then  stand  by  for  a  lengthy  disser¬ 
tation  on  every  design  detail.  Ask  that  same  manager  a  software  related 
question  and  you  will  get  the,  "things  are  right  on  schedule"  answer.  Realiz¬ 
ing  that  this  whole  software  reliance  environment  is  a  relatively  new  field,  it 
will  still  take  some  time  before  a  true  appreciation  of  the  software  design  pro¬ 
cess  is  attained. 

Watts  S.  Humphrey,  of  the  Software  Engineering  Institute,  outlines  six 
common  misconceptions  about  the  software  process.  He  lists: 

•  Wie  must  start  with  firm  requirements.  There  is  a  widespread  but 
fallacious  view  that  requirements  are  the  customers’  job  and  that 
development  should  not  start  until  they  are  explicitly  defined. 

•  If  it  passes  test,  it  must  be  OK.  If  the  generally  dismal  record  of 
software  quality  problems  doesn’t  prove  this  false,  nothing  will. 

•  Software  quality  can't  be  measured.  While  not  as  well  recognized, 
this  too  is  false. 

•  The  problems  are  technical.  In  spite  of  the  many  improved  lan¬ 
guages,  tools,  and  environments,  the  problems  of  software  cost, 
schedule,  and  quality  remain. 

•  We  need  better  people.  Since  software  professionals  make  the  errors, 
some  people  erroneously  feel  that  they  should  be  blamed  for  them. 

•  Software  management  is  different.  While  this  is  a  new  and  unique 
field,  traditional  management  methods  can  and  should  be  used. 
[Ref  43:  25] 

Mr.  Humphrey  further  states  that,  "The  remarkable  thing  about  these  mis¬ 
conceptions  is  that  even  though  most  software  professionals  recognize  their 
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fallacy  our  contracts  and  our  management  systems  are  largely  based  on 
them."  [Ref.  43:25] 

As  stated  earlier,  although  recognized  as  the  most  effective  and  efficient 

way  to  develop  software  to  meet  new  requirements,  the  prototyping  approach 

basically  conflicts  with  contracting  practices.  One  of  the  things  that  Mr. 

Humphrey  has  done  is  to  look  at  the  mind-set  of  those  in  the  procurement 

business.  He  looks  at  software  contracting  in  terms  of  trust. 

How  would  buyers  behave  if  they  did  not  trust  their  vendors,  doubted 
their  competence,  and  believed  in  their  own  ?  Under  these  conditions, 
they  would  probably  do  the  following: 

1.  Start  with  a  very  precise  statement  of  what  was  wanted. 

2.  Insist  on  rigid  standards  and  detailed  documentation  of  every 
step. 

3.  Require  that  each  step  be  completed  and  approved  before  the 
next  was  initiated. 

4.  Demand  a  firm  commitment  at  the  outset.  [Ref.  43:  411] 

Humphrey  is  not  implying  that  software  developers  are  dishonest  liars, 
cheaters  or  stealers,  but  rather,  that  trust  is  a  relational  attitude  between 
program  managers  and  contracting  officers  and  the  contractor.  Like  in  any 
other  relationship,  trust  is  slow  to  evolve  and  quick  to  evaporate.  Trust  is 
developed  over  time  as  the  two  parties  become  familiar  with  each  other. 
Humphrey,  on  the  other  hand,  warns,  "while  trust  is  important,  blind  faith  is 
not.  There  is,  however,  an  enormous  difference  between  an  adversarial  con¬ 
tract  and  one  with  a  reasonable  level  of  arm’s-length  protection"  [Ref.  43: 
413]. 

This  basic  principal  of  trust  readily  applies  in  the  software  contracting 
environment.  It  is  unfortunate,  however,  that  the  entire  software  develop¬ 
ment  environment  is  judged  on  those  instances  where  success  is  not 
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optimxim.  In  other  words,  successful  software  development  programs  are  not 
news  worthy.  On  the  other  hand,  as  highlighted  in  Chapter  II,  the  "not  so 
successful"  development  programs  consume  such  a  tremendous  amotmt  of 
resources  that  all  programs  tend  to  be  approached  with  skepticism.  This 
skepticism  can  have  a  positive  benefit  if  it  results  in  a  management,  inspec¬ 
tion,  quality,  and  test  program  that  will  audit  developmental  efforts.  If  done 
properly,  the  results  wiU  be  an  environment  of  trust,  the  win/win  relationship 
that  we  strive  to  achieve. 

C.  SOFTWARE  QUALITY 

In  Chapter  I,  quality  was  equated  to  the  software's  ability  to  meet  the 
user's  ultimate  mission  need.  Although  from  the  user's  perspective  this 
approach  may  be  adequate,  from  a  management  perspective  it  is  much  too 
simplistic.  The  program  manager  and  contracting  officer  must  be  able  to 
track  the  program  through  all  stages  of  development.  Management's  assess¬ 
ment  of  the  development  program  cannot  be  based  solely  on  the  contractor's 
ability  to  satisfy  user  requirements  and/or  meet  scheduled  milestones.  This 
approach  would  imply  that  the  manager  might  not  have  a  clue  as  to  the  real 
status  of  the  software  development  program  until  the  end  product  is  deliv¬ 
ered.  It  should  be  obvious  that  this  "wait  and  see"  approach  is  not  very  wise 
considering  the  consequences  of  a  embedded  software  development  program 
that  suffers  cost  ovemrns  and  schedule  delays.  Software  quality  must  there¬ 
fore  be  defined  in  terms  other  than  customer  satisfaction.  It  is  difficult,  at 
best,  to  define  and  quantify  measures  of  software  quality. 

Measures  of  quality  can  be  grouped  into  five  general  classes:  Develop¬ 
ment,  Product,  Acceptance,  Usage,  and  Repair.  Table  VII  [Ref.  43:  339] 
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TABLE  VII,  Classes  of  Quality  Measures 


Objsclive 

Ttmely 

Available 

Representative 

Controllable 

Development 

Defects 

yes 

yes 

yes 

moderate 

yes 

Change  activity 

yes 

yes 

yes 

poor 

no 

Product 

Error  seeding 

moderate 

yes 

difficult 

doubtful 

moderate 

Software  structure 

depends 

yes 

moderate 

doubtful 

yes 

Controlled  tests 

mr^erate 

yes 

difficult 

good 

yes 

Acceptance 

Problems 

no 

late 

yes 

good 

moderate 

Install  effort 

moderate 

late 

difficult 

good 

yes 

Usage 

Problems 

no 

late 

yes 

good 

moderate 

Operating  effort 

moderate 

late 

difficult 

good 

yes 

Sun/eys 

no 

late 

difficult 

very  good 

no 

Availability 

yes 

late 

moderate 

very  good 

yes 

Repair 

Defects 

yes 

late 

yes 

moderate 

yes 

Repair  effort 

moderate 

late 

moderate 

moderate 

yes 

Source:  Humphrey,  Watts  S.  Managing  the  Software  Process 


depicts  a  matrix  that  characterizes  of  each  of  these  categories  with  respect  to 
objectivity,  timeliness,  availability,  the  degree  to  which  the  buyer  believes  it 
is  good  (representation),  and  controllability.  The  program  manager  must 
choose  those  criteria  which  will  best  serve  the  program's  needs.  He  must  then 
translate  those  criteria  into  contractual  requirements.  Those  contractual 
requirements  must,  in  turn,  be  in  such  a  format  and  language  as  to  allow 
their  effective  use.  The  program  manager  will  undoubtedly  have  access  to 
technical  assistance,  but  simplicity  and  ease  of  use  by  nontechnical  decision 
makers  will  remain  the  key  to  useful  management  measurements/tools.  The 
intent,  here,  is  not  to  provide  a  detailed  explanation  of  each  of  the  criteria.  It 
is,  however,  to  show  that  there  are  many  criteria  with  varying  degrees  of 
appropriateness  depending  on  the  nature  of  the  program  for  measuring 
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software  quality.  Each  of  the  criteria  can  play  a  vital  role  in  measuring 
program  success.  For  example,  one  of  the  quality  measuring  criteria  calls  for 
the  monitoring  of  defects  during  development.  Commonly  experienced  error 
rates  between  1  and  3  errors  per  1000  lines  of  code  (kloc)  seem  somewhat 
trivial,  and  in  the  past  somewhat  acceptable.  Today's  increased  reliance  on 
software  to  satisfy  weapon  systems'  mission  requirements,  however,  make 
these  numbers  unacceptable.  It  is  estimated  that  the  Strategic  Defense 
Initiative  (SDI)  program  will  total  over  25  million  lines  of  code.  Applying  the 
1-3/kloc  error  rate  would  mean  between  25,000  and  75,000  errors  in  the 
operational  and  support  program  [Ref.  44:  93-94].  Management  attention 
must  be  turned  toward  the  process,  not  just  the  result.  "To  consistently 
achieve  superior  performance,  management  must  establish  challenging 
quality  goals  and  strive  to  meet  them"  [Ref.  43:  358]. 

D.  MANAGEMENT  ASSESSMENTS 

The  first  step  in  developing  the  relationship  of  trust,  as  previously  dis¬ 
cussed,  is  to  establish  a  baseline  with  regard  to  a  contractor's  capabilities. 
Taking  the  Total  Quality  Management  (TQM)  approach,  this  means  not  only 
an  assessment  of  the  contractor’s  capabilities,  but  more  importantly,  an 
assessment  of  his  organizational  software  development  process.  Quality  is  a 
direct  reflection  of  the  company's  process.  A  concentrated  effort/pressure 
should  be  applied  on  the  contractor  to  improve  his  own  process,  thus,  result¬ 
ing  in  an  improvement  to  quality.  The  Software  Engineering  Institute  (SEI), 
Carnegie  Mellon  University,  has  issued  two  technical  reports  that  can  aid  the 
program  manager  during  this  assessment. 
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1.  A  Method  for  Assessing  the  Software  Engineering  Capability 
of  Contractors.  (CMU/SEI-87-TR-23) 

The  purpose  of  this  document  is  to,  "facilitate  objective  and  consis¬ 
tent  assessments  of  the  ability  of  potential  DOD  contractors  to  develop  soft¬ 
ware  in  accordance  with  modem  engineering  methods"  [Ref.  45:  1].  Requested 
by  the  U.S.  Air  Force,  this  effort  was  a  result  of  the  realization  that  the 
government  must  be  able  to  effectively  evaluate  the  capabilities  of  potential 
contractors.  An  extremely  straight  forward  docximent,  it  provides  a  series  of 
questions  to  evaluate  the  contractor's  software  development  process.  It  is  an 
outstanding  tool  for  every  program  manager,  contracting  officer,  and  source 
selection  participant.  The  intent  of  this  program  is  to  provide  potential  con¬ 
tractors  with  a  questionnaire,  based  on  this  document.  An  assessment  team  is 
then  sent  to  each  of  the  contractors'  facilities  to  address  each  of  the  questions 
in  detail  collecting  data  to  make  their  final  evaluation. 

2,  Conducting  SEI-Assisted  Software  Process  Assessments 
This  document  describes  the  Software  Engineering  Institute's  pro¬ 
cedure  for  assessing  a  contractor's  software  process.  It  is  not  intended  as  a 
guide  that  can  be  used  by  the  program  manager,  it  does  however,  provide  a 
description  of  the  Software  Engineering  Institute’s  capability  to  review  an 
organization's  software  process.  The  assessment  will  determine  the  software 
process  that  is  being  used,  identify  key  areas  for  improvement,  and  provide 
the  contractor  with  assistance  for  implementing  and  monitoring  the  changes. 
While  this  level  of  effort  may  not  be  appropriate  for  all  programs,  the  pro¬ 
gram  manager  should  consider  an  early  assessment  of  the  contractor's  pro¬ 
cess.  For  many  of  the  major  development  programs,  this  guide  would  be 
excellent.  [Ref  46] 


43 


E.  DEPARTMENT  OF  DEFENSE  REPORT  ON  MILITARY 

SOFTWARE 

In  iate  1987,  the  Defense  Science  Board  Task  Force  on  Military  Software 

forwarded  its  report  to  the  Secretary  of  Defense.  The  Board  reported  on  the 

various  initiatives  within  the  Department  of  Defense  aimed  at  improving 

software  development.  They  included  the  Ada  programming  standard,  the 

Software  Technology  for  Adaptable,  Reliable  Systems  (STARS),  the  Strategic 

Computing  Initiative,  the  Software  Engineering  Institute,  and  the  Strategic 

Defense  Initiative,  the  Executive  Summary  of  this  report  stated: 

...The  big  problems  are  not  technical.  In  spite  of  the  substantial  techni¬ 
cal  development  needed  in  requirements- setting,  metrics  and  mea¬ 
sures,  tools,  etc.,  the  Task  Force  is  convinced  that  today's  major 
problems  with  military  software  development  are  not  technical,  but 
management  problems... [Ref.  47: 1] 

F.  DEPARTMENT  OF  DEFENSE  SOFTWARE  MASTER  PLAN 

One  step  in  the  right  direction  is  the  publication  of  the  Department  of 
Defense's  "Software  Master  Plan".  Released  in  preliminary  draft  form  during 
February,  1990,  it  is  the  Department  of  Defense's  first  effort  to  recognize  and 
bring  to  light  the  unique  problems  associated  with  military  software  devel¬ 
opment.  It  begins  with  a  description  of  the  various  activities  that  have  a 
management  role  in  software  procurement.  It  begins,  "In  order  to  identify 
actions  required  to  improve  the  software  management  process,  an  under¬ 
standing  of  the  current  software  management  roles... within  the  DOD  is 
essential"  [Ref  25:  A-1].  The  significant  issue  here  is  the  realization  that  the 
"software  management  process”  is  unique  entity,  unlike  hardware  develop¬ 
ment,  and  the  current  situation  requires  top  level  management  attention. 
Specific  direction  to  improve  the  "process"  is  not  provided,  however,  it  can  be 
easily  deduced  that  this  document  will  provide  the  basis  for  corrective  action 
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within  the  DOD.  It  states,  "...as  evidenced  by  the  information  presented... the 
overall  responsibility  for  software  management  is  clearly  fragmented  across 
the  DOD"  [Ref  25:  A-1]. 

G.  SUMMARY 

This  chapter  has  provided  a  broad  overview  of  the  many  complex  factors 
that  a  program  manager  must  be  aware  of  while  managing  a  program  involv¬ 
ing  software  development.  There  must  exist  an  awareness  of  the  impacts  of 
philosophical,  environmental,  and  quality  issues.  Addressing  these  issues 
during  the  acquisition  strategy  formulation  is  a  must. 

The  intent  of  this  chapter  is  to  show  that  even  though  the  contracting 
world  seems  somewhat  rigid  and  inflexible,  there  are  many  approaches  that 
can  be  taken  to  improve  software  quality  without  having  to  change  the  con¬ 
tracting  process.  The  significance  is  an  emphasis  on  a  change  of  the  software 
development  process,  and  not  the  contracting  process. 
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V.  DATA  COLLECTION  RESULTS 


A.  DATA  COLLECTION  SURVEY 

Having  completed  an  extensive  review  of  the  literature  available,  the 
next  step  was  to  see  exactly  how  this  "software  development  problem"  is  per¬ 
ceived  in  the  field.  In  addition  to  the  personal  interviews  that  were  conducted, 
a  survey  was  forwarded  to  150  management  and  contracting  professionals 
from  both  the  government  and  industry.  Of  the  65  responses  received,  49 
reported  having  had  experience  in  programs  that  involved  the  development 
and  acquisition  of  mission  critical  software.  The  average  experience  level  of 
the  respondents  was  9.5  years,  ranging  from  30  years  of  experience  to  one 
year.  A  sample  survey  form  is  contained  in  Appendix  A.  Compiled  survey 
results  are  contained  in  Appendix  B. 

There  was  a  overwhelming  consensus  that  a  problem  does  indeed  exist 
in  the  government's  ability  to  procure  custom  designed  software.  The  survey 
responses  indicated  that  each  of  the  three  areas,  technical,  contracting,  and 
program  management  equally  shared  the  blame  for  the  problems  encoun¬ 
tered.  The  following  sections  discuss  each  of  the  major  survey  areas. 

B.  PROBLEMS  WITH  SOFTWARE  ACQUISITION 

Respondents  were  asked  to  identify  the  percentage  of  software  develop¬ 
ment  programs  that  resulted  in  cost  overruns,  schedule  delays  and  poor 
performance  due  to  problems  in  one  of  three  general  areas:  technical,  contrac¬ 
tual,  managerial.  Table  VIII  provides  typical  examples  for  each  of  the  three 


TABLE  VIII.  Three  Categories  of  Problems  in  Software  Development 


AREA 

EXAMPLE 

TECHNICAL 

Programming  or  interface  difficulties 

CONTRACTUAL 

Contract  interpretation  difficulties, 
contract  administration  difficulties,  or 
possibly  the  use  of  an  inappropriate 
contract  type. 

MANAGERIAL 

Improper  acquisition  strategies,  bad 
business  decisions,  lack  of  attention, 
lack  of  planning 

problem  categories.  For  the  purposes  of  this  particular  question,  the  respon¬ 
dents  were  also  divided  into  three  categories:  technical  personnel,  contracting 
personnel,  and  management  personnel. 

1.  Technical  Problems 

The  survey  showed  a  perception  exists  that  59  percent  of  software 
development  programs  suffered  delays,  cost  overruns,  and  performance  prob¬ 
lems  due  to  technical  problems.  Managers  felt  that  63.4  percent  of  the  pro¬ 
grams  had  technical  problems,  while  the  technicians  themselves  seemed  to 
think  it  was  more  like  56  percent.  Contracting  personnel  were  right  at  the  59 
percent  level. 

2.  Contracting  Problems 

Overall,  the  respondents  felt  that  46.5  percent  of  software  develop¬ 
ment  programs  suffered  problems  due  to  contracting  issues.  Technical  and 
contracting  personnel  agreed  that  48  percent  of  the  programs  were  affected 
by  contracting  issues.  Management,  on  the  other  hand,  felt  that  only  42 


47 


percent  of  the  programs  suffered  delays,  cost  overruns,  and  performance  prob¬ 
lems  due  to  contracting  issues. 

3.  Program  Management  Problems 

Managers  blew  the  whistle  on  themselves.  They  felt  that  61.2  per¬ 
cent  of  the  programs  encountered  problems  due  to  "management  related  prob¬ 
lems."  The  overall  perception  was  57.7  percent.  Contracting  personnel 
believed  that  only  50  percent  of  the  programs  suffered  problems  due  to  man¬ 
agement  issues,  while  the  technical  folks  felt  it  was  almost  59  percent. 

4.  Summary 

Based  on  the  data  collected  in  this  section  of  the  survey,  there  is  a 
perception  that  software  development  programs  encounter  delays,  overruns, 
and  performance  problems  almost  60  percent  of  the  time  due  to  technical 
issues,  almost  58  percent  of  the  time  due  to  managerial,  and  less  that  47  per¬ 
cent  of  the  time  due  to  contracting  issues.  A  complete  listing  of  the  responses 
collected  is  contained  in  Appendix  B. 

C.  TOP  FIVE  CONTRIBUTIONS  TO  DEVELOPMENT  PROBLEMS 

The  next  section  of  the  survey  asked  respondents  to  identify  the  five, 
"...most  common  problems  regarding  contracting  for  and  the  management  of 
a  program  involving  software  development".  Table  IX  is  a  list  of  alternatives 
that  were  contained  in  the  questionnaire.  Answers  were  compiled  using  a 
simple  weighting  system  to  rank  the  responses.  The  ranking  system  assigned 
a  weight  of  10  points  to  the  number  one  response,  eight  to  number  two,  six  to 
number  three,  four  to  number  four,  and  two  points  to  the  number  five 
response.  The  survey  overwhelmingly  showed  that  the  respondents  believed 
that  the  government's  inability  to  identify  clear  user  reqmrements  was  the 


TABLE  IX.  Most  Common  Problem  Areas  in  Software  Development 


Unclear  user  requirements 
Schedule  too  optimistic 
Lack  of  govt  regulations 
Inadequate  govt  specifications 
Lack  of  timely  user  feedback 
Wrong  personnel  at  design  reviews 
Lack  of  software  design  freeze 
Failure  to  designate  POC's 
Improper  contract  type 
Lack  of  qualified  govt  tech  personnel 
Too  many  changes  to  requirements 
Inability  to  estimate  cost 
Lack  of  understanding  regarding 
software  design  and  development 
Mismanagement  by  industry 


Unclear  statem^int  of  work 
Contract  clauses  ioo  restrictive 
Too  many  govt  regulations 
Too  many  govt  specifications 
Lack  of  design  reviews 
Lack  of  hardware  design  freeze 
Inadequate  testing  (govt) 
Inadequate  testing  (contractor) 
Difficult  acquisition  procedures 
Inherent  inability  to  measure 
reliability  of  software 
Lack  of  contractor  incentive 
Lack  of  adequate  documentation 
Lack  of  configuration  mgmt 
Mismanagement  by  govt 
Others 


number  one  problem  in  software  development.  The  top  five  answers,  along 
with  their  weightings  are  listed  in  Table  X.  A  complete  listing  of  all  responses 
can  be  found  in  Appendix  B. 


TABLE  X.  Five  Most  Common  Problems  in  Software  Development 


RACKING 

PRQBLENl 

TOTAL  POINTS 

1. 

UNCLEAR  USER  REQUIRE»fENTS 

252 

2. 

TOO  MANY  CHANGES  TO  REQUIREMENTS 

140 

3. 

LACK  OF  UNDERSTANDING  REGARDING  SOFTWARE 
DESIGN  AND  DEVELOPMENT 

134 

4. 

SCHEDULE  TOO  OPTIMISTIC 

130 

5. 

UNCLEAR  STATEMENT  OF  WORK 

92 

49 


1.  Unclear  User  Requirements 

As  for  what  the  respondents  felt  was  at  the  heart  of  the  num¬ 
ber  one  problem,  impressions  varied  greatly.  "Users  tend  to  not  know  what 
they  want  til  they  see  it  half  done,  then  they  are  sure  they  want  something 
different",  said  one  respondent.  On  the  other  end  of  the  spectrum,  another  felt 
that,  "although  the  customer  knows  what  he  wants,  he  is  unable  to  communi¬ 
cate  his  needs  to  the  contractor..."  Mr.  Ted  Bosworth,  Vice  President  for  C3 
Systems  at  Comptek  Research  in  Buffalo,  NY,  summed  up  the  problem  by 
saying,  "The  typically  inaccurate  initial  translation  between  user  and  engi¬ 
neering  communities,  and  the  years  between  initial  statement  of  need  and 
completion  of  development,  virtually  assures  that  software  'forks'  are  deliv¬ 
ered  to  provide  users  the  ability  to  eat  their  'soup'"  [Ref.  48].  No  matter  what 
the  underl3ring  reason,  a  lack  of  knowledge  or  a  lack  of  commimication,  it 
seems  only  logical  that  the  government  must  develop  a  means  to  better 
identify  end  user  requirements.  Finally,  one  respondent  stated,  "Too  many 
times  the  wrong  people  are  asked  what  is  required.  If  we  could  produce  a 
rapid  prototype  to  touch,  feel  and  see  maybe  we  would  do  a  better  job  of  it." 

2.  Too  Many  Changes  to  Requirements 

Given  that  the  government  has  problems  identifying  its  require¬ 
ments  lends  to  the  second  problem  area  identified  by  survey  respondents,  "too 
many  changes  to  requirements."  On  this  subject  one  respondent  said,  "It  is 
common  to  have  several  ECP's  in  work  at  several  times  all  at  different  stages 
of  definition,  design,  development  or  incorporation.  This  leads  to  multiple 
baselines  which  complicates...  configuration  management,  schedules,  esti¬ 
mating,  and  program  management  by  both  industry  and  government." 
Another  expressed  the  difficulties  with  too  many  changes  by  making  the 
analogy  of  trying  to  "hit  a  moving  target." 


3.  Lack  of  Understanding  Regarding  Software  Design  and 

Development 

Addressing  this  issue  one  respondent  said: 

We  (Govt)  are  not  careful  enough  in  our  monitoring  of  processes  that 
result  in  products/services.  We  traditionally  have  viewed  (inspected) 
the  finished  product...  In  the  case  of  software,  this  lack  of  attention  to 
processes  in  design  and  development  may  fatally  flaw  the  finished 
product  and  yet  the  flaw  may  go  undetected  for  months/years. 

There  seems  to  be  a  consensus  among  respondents  who  chose  this  issue  as 
one  of  their  top  five.  One  respondent  felt  that,  "...almost  everyone  THINKS 
they  understand  software  design  and  development  and  yet  they  do  not  com¬ 
municate  v  eil  with  their  counterparts  in  industry/govemment."  Another 
respondent  felt  that,  "issues  such  as  'Design  for  Supportability'  and  compati¬ 
bility  or  interoperability  get  confused  with  ownership  and  costyprofit  issues." 
It  seems  that  this  issue  once  again  stresses  the  need  for  open  lines  of  commu¬ 
nication  between  the  government  and  industry. 

4.  Schedule  Too  Optimistic 

A  survey  respondent,  a  program  manager  in  the  private  sector,  who 
listed  this  as  his  number  one  problem  said,  "Typically  the  government  acqui¬ 
sition  process  is  so  slow  that  it  becomes  mandatory  to  accelerate  the  devel¬ 
opment  process  in  order  to  meet  user  needs.  The  complexity  of  the  task  is  not 
re-scoped  and  this  forces  the  contractor  into  attempting  to  achieve  imrealistic 
schedules."  Mr.  Ted  Bosworth  stated: 

Every  program  I  have  been  involved  with  that  had  significant  software 
content  had  a  schedule  that  assvimed  the  user  community  knew  exactly 
what  they  wanted;  the  Government  development  activity  translated 
operational  requirements  into  a  technical  specification  completely, 
accurately,  and  without  the  possibility  of  misinterpretation  by  devel¬ 
opment  engineers,  and  the  engineers  made  no  mistsJces.  Unfortunately 
for  schedule  performance,  I  have  yet  to  see  a  set  of  humans  that  good. 
[Ref  48] 
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5.  Unclear  Statement  of  Work 

Many  of  these  top  five  problems  are  interrelated.  If  the  users  fail  to 
identify  their  needs  accurately,  then  how  can  the  program  office  write  an 
adequate  statement  of  work?  One  respondent  stated,  "Initial  SOW s  are  what 
drive  proposals  —  not  what  becomes  the  end  item  other  than  as  an  overall 
skeletal  shell.  The  upgrading  of  the  SOW  is  often  not  accomplished  in  a 
timely  manner:  hence  there  is  an  attempt  to  maintain  proposal  schedules 
which  no  longer  reflect  the  realities."  Another  felt  that,  "many  times  the 
Statement  of  Work  details  are  not  clearly  detailed  in  writing  and  it  easily 
leads  to  misunderstandings  on  the  part  of  the  Contractor."  On  the  other 
hand,  one  respondent  felt  that,  "an  imclear  statement  of  work  isn't  necessar¬ 
ily  all  bad  —  especially  with  Full  Scale  Development  programs...  more 
emphasis  should  be  placed  on  what  the  contractor  intends  to  do  and  how  they 
priced  it."  Still  another  respondent  said,  "...SOWs  often  have  both  too  much 
detail  and  not  enough.  Too  much  detail  limits  the  creativity  of  the  contractor, 
and  not  enough  results  in  a  program  that  doesn't  meet  the  need."  In  speaking 
about  the  contents  of  a  statement  of  work,  one  respondent  said,  "Too  many 
technical  requirements  are  levied  in  lieu  of  user  requirements,  let  the  user 
define  the  requirements  not  the  engineers  interpreting  the  user's 
requirements." 

D.  CONTRACrmO  FOR  SOFTWARE 
1.  Selection  of  a  Contract  T3rpe 

As  discussed  earlier  in  Chapter  III,  the  selection  of  a  contract  type 
is  a  discretionary  decision  of  the  contracting  officer  and  program  manager, 
with  some  room  for  negotiation  with  the  contractor.  It  is  a  decision  that  will 
hopefully  result  in  a  win-win  situation  between  the  government  and  the 


contractor.  Although  many  factors  are  involved  in  the  decision  process,  the 
major  factor  is  usually  a  question  of  risk  assumption.  Typically  development 
programs,  considered  somewhat  of  a  higher  risk,  would  use  some  type  of  cost 
reimbursement  arrangement.  The  opposite  holds  true  in  a  more  stable  envi¬ 
ronment  such  as  a  follow-on  or  production  effort  where  a  fixed  price  arrange¬ 
ment  is  typically  used. 

One  of  the  early  premises  of  this  thesis  was  that  contract  type  might 
play  a  significant  role  in  the  eyes  of  the  personnel  involved  in  the  procure¬ 
ment  of  custom  designed  software.  This  was  shown  to  be  imtrue.  Only  three 
of  the  49  respondents  considered  "improper  contract  type"  as  one  of  their  top 
five  concerns  in  software  development  programs.  One  respondent  summed  up 
this  issue  by  stating; 

The  contract  type  is  not  the  key  to  success.  A  good  contract  is  worth 
crap  if  the  adii^n  is  poor.  A  bad  contract  can  be  a  success  if  adminis¬ 
tered  well.  The  problem  is  not  the  contract  type  it’s  how  to  make  it 
work  to  meet  your  need.  (Anonymous) 

2.  Advantages  and  Disadvantages  of  Contract  Types 

The  two  general  categories  of  contract  types  that  are  available  for 
software  development  are  cost  reimbursement  and  fixed  price.  Each  has  its 
advantages  and  disadvantages  along  with  their  appropriate  uses. 

It  may  be  very  appropriate  to  use  a  firm  fixed-price  contract  for 
easily  and  accurately  estimated  software  efforts.  The  biggest  advantage  of  a 
firm  fixed-price  contract  is  there  is  very  little  administrative  overhead.  The 
contractor  assumes  the  technical,  schedule  and  cost  risk.  Once  the  contract  is 
awarded  the  only  concern  becomes  the  deliverable  product.  On  the  other 
hand,  a  firm  fixed-price  contract  may  be  the  most  inappropriate  contract  to 
use  for  software  development,  such  as  a  new  start  weapons  program. 
This  type  of  pricing  arrangement  offers  almost  no  flexibility  to  either  the 
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contractor  or  the  government.  In  a  development  environment,  this  means  that 
the  product  is  built  to  the  specifications  in  the  contract,  period.  If,  as 
discussed  above,  the  users  have  problems  communicating  their  needs,  the 
program  office  has  problems  interpreting  their  needs,  and  the  engineers  have 
problems  satisfjring  those  needs,  the  government  and  the  contractor  are 
locked  into  an  agreement  that  isn’t  worth  the  paper  its  written  on.  The 
contractor  is  not  incentivized  to  seek  new,  innovative,  and  possibly  more 
efficient  ways  to  satisfy  software  requirements.  Likewise,  the  contractor  has 
little  motivation  to  take  the  extra  time  required  to  document  the  software 
other  than  the  minimal  requirements  tmder  the  FFP  agreement.  This  may 
lead  to  tremendous  maintenance  troubles  during  the  systems  life  cycle. 
Contractors  on  a  tight  schedule  and/or  budget  are  less  likely  to  restrict  the 
level  of  effort  put  in  to  systems  development.  Most  corporations  would  be 
very  leery  to  sign  a  firm  fixed  price  contract  for  those  development  programs 
that  contain,  in  a  prudent  businessman’s  judgment,  too  many  uncertainties. 

The  other  option  available  for  a  software  development  effort  is  a 
cost  reimbursement  pricing  arrangement.  Under  this  arrangement,  the  con¬ 
tractor  is  relieved  of  the  cost  risk  associated  with  a  development  program. 
The  contractor  has  some  flexibility  as  to  how  to  satisfy  the  user’s  needs,  and 
is  not  locked  in  to  the  single  approach  selected  at  the  onset  of  the  program. 
The  major  disadvantage  lies  in  the  fact  that  the  contractor  is  entitled  to  all 
allowable  and  allocable  expenses  incurred.  This  means  that  the  program 
office  may  have  imderestimated  the  task  in  terms  of  cost  and  time,  resulting 
in  a  need  for  additional  funding.  Cost  reimbursement  contracts,  in  compari¬ 
son  to  fixed  price  arrangements,  require  a  great  deal  of  administrative 
overhead. 


3.  Recommendations  from  the  Field 

Survey  respondents  were  asked  to  identify  the  contract  types  that 
they  had  worked  with  during  a  software  development  effort.  Responses 
covered  the  entire  spectrum  of  contract  types,  bar  none,  outlined  in  the  FAR. 
Survey  respondents  were  then  asked  to  identify  the  contract  type  that,  "oflfers 
the  most  benefit  to  both  the  government  and  the  contractor  during  a  software 
development  effort."  A  list  of  the  recommended  contract  types  is  shown  in 
Table  XI.  Eighteen  of  the  respondents  felt  that  they  could  not  give  an  answer 
to  the  question.  Most  said  that  it  was  impossible  to  generalize,  and  the  selec¬ 
tion  of  contract  type  depended  on  the  particular  situation  for  each  individual 
program. 


TABLE  XI.  Recommended  Contract  Types  for  Software  Development 


NO  PARTICULAR  RECOMMENDATION  18 

COST-PLUS-AWARD-FEE  10 

FIXED-PRICE-INCENTIVE  8 

COST-PLUS-FIXED-FEE  6 

COST-PLUS-INCENTIVE  3 

COST  TYPE  (UNSPECIFIED)  2 

FIRM-FIXED-PRICE  2 


4.  Complicating  the  Issue 

It  is  very  unlikely  that  a  contract  for  the  development  of  embedded 
mission  critical  software  will  be  segregated  from  the  hardware  development 
effort.  The  selection  of  contract  type  now  becomes  a  question  of  which  con¬ 
tract  will  best  suit  the  needs  of  the  entire  program. 


55 


If  an  incentive  type  contract  is  selected,  the  government  must  now 
decide  just  what  to  incentivize.  Remember  what  was  said  about  hardware 
earlier,  we  can  see  and  feel  it.  It  is  much  easier  to  structure  an  incentive  con¬ 
tract  aroiind  those  things  that  can  be  seen  and  felt.  This  leaves  a  very  diffi¬ 
cult  task  for  the  contracting  officer  and  program  manager  who  attempt  to 
structure  the  incentive  t3rpe  contract  around  software,  as  well  as,  hardware 
development  measures. 

It  is  not  impossible  to  use  incentive  contracts  for  software  develop¬ 
ment.  It  does,  however,  require  a  great  deal  of  administrative  and  manage¬ 
ment  effort.  In  an  interview  with  Mr.  James  Swizewski,  a  contracting  officer 
from  the  Naval  Regional  Contracting  Center  in  Philadelphia,  the  subject  of 
using  a  CPAF  contract  was  discussed.  He  stated  that,  in  fact,  NRCC  had  used 
a  CPAF  contract  for  software  development.  He  did  not  feel,  however,  that  the 
contract  was  executed  efficiently  [Ref.  49].  Even  though  the  contractual  vehi¬ 
cle  was  in  place,  the  contractor  was  not  incentivized  appropriately.  A  major 
draw  back  to  the  use  of  a  CPAF  contract  is  that  the  contractor  will  concen¬ 
trate  his  efforts  on  those  measures  in  the  contract  that  will  result  in  his 
receiving  the  award  fee.  In  other  words,  regardless  of  those  areas  that  may 
require  special  attention  based  on  events  during  the  contract  performance, 
attention  will  go  to  areas  defined  at  contract  award  that  result  in  the  receipt 
of  the  award  fee.  Contractors  have  been  known  to  devote  so  much  attention  to 
the  award  fee  criteria  that  the  rest  of  the  program  suffers.  Another  drawback 
is  the  tremendous  requirement  for  administrative  and  managerial  overhead 
on  the  part  of  the  government.  Mr.  Bud  Wasgatt,  Division  Head  of  the  Navy's 
Combat  Systems  Test  and  Analysis  Department,  Port  Hueneme,  CA,  felt  that 
the  use  of  a  CPAF  contract  is  viable,  but  that  the  award  fee  must  be  tied  to 
some  software  quality  indicator  [Ref.  50].  There  lies  the  problem.  Again,  we 


56 


are  back  at  the  point  of  trying  to  identify  and  define  software  performance 
indicators. 


E.  INDEPENDENT  VERIFICATION  AND  VALIDATION 

One  of  the  tools  available  to  the  program  office  is  the  use  of  an  Indepen¬ 
dent  Verification  and  Validation  (IV&V)  organization  to  monitor  the  software 
development  process.  The  IV&V  personnel  are  the  government's  coimterparts 
to  the  company's  Software  Quality  Assurance  (SQA)  staff.  Mr.  Htunphrey 
best  states  the  concept  of  IV&V: 

...rV&V  can  and  should  capitalize  on  the  existence  of  SQA.  If  SQA 
is  working  effectively,  IV&V  need  not  duplicate  its  work,  and  if  not, 
IV&V  must  not  try  to  replace  it.  Their  role  is  to  highlight  this  short¬ 
coming  and  get  it  fixed,  [^f.  43: 151] 

Duties  of  the  IV&V  staff  include:  design  and  coding  analysis,  approval  of 
tests  and  test  procedures,  and  witnessing  of  tests  and  inspections.  Mr. 
Jimmie  Carlisle,  Atlantic  Research  Corporation,  serves  as  the  head  of  the 
rV^&V  team  for  the  Marine  Corps  at  Litton  Data  Systems  Division  in  Los 
Angeles.  Litton  has  developed  and  is  currently  producing  the  AN/TYQ-23, 
Tactical  Air  Operations  Module,  for  the  Marine  Corps  and  the  U.S.  Air  Force. 
He  feels  that  the  money  dedicated  up  front  by  the  Marine  Corps  to  place  his 
rV&V  team  in-plant  will  result  in  tremendous  payback  during  the  software 
operational  and  maintenance  life  cycle.  Mr.  Carlisle  also  sees  an  expansion  of 
the  role  of  IV&V  in  the  future  to  include  some  types  of  independent  testing. 
[Ref.  51] 

The  Marine  Corps'  SQA  representative  at  the  Marine  Corps  Tactical 
Systems  Support  Activity  (MCTSSA),  located  at  Camp  Pendleton,  CA,  whole¬ 
heartedly  concurs.  CWO-3  David  Mrazik  feels  that  it  is  absolutely  essential 
to  dedicate  resources  during  the  development  phase,  independent  of  the 
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developing  activity,  to  ensure  supportability  and  maintainability  throughout 
the  software's  life  cycle.  MCTSSA  will  assume  software  maintenance 
responsibility  for  the  AN/TYQ-23  and  its  some  3.7  million  lines  of  code 
(estimated)  in  1992.  [Ref.  52] 

F.  PERFORMANCE  WARRANTIES 

Another  way  that  the  government  can  insure  itself,  at  least  against 
monetary  impacts,  is  to  negotiate  a  warranty  as  part  of  the  development/ 
production  contract.  Most  typically  used  in  the  case  of  hardware  deliverables, 
warranties  protect  against  patent  defects  after  government  acceptance. 
(Patent  defects  are  defects  which  should  have  been  found  through  normal 
inspection.)  Adding  a  slight  twist  to  this  concept,  program  offices  are  now 
using  warranties  to  protect  against  software  defects.  The  AN/TYQ-23  is  such 
a  program.  A  performance  warranty  is  included  in  the  contract  stating  that: 

The  TACM  Computer  Programs  which  reside  in  the  AN/AYK-14 
or  AN/UYK-44  Computer  and  the  software/firmware  embedded  in  the 
various  unit  processors... shall  be  free  of  performance  defects  during 
the  period  specified  under  PERFORMANCE  WARRANTY  PERIOD.  A 
performance  defect  is  defined  as  a  failure  of  the  computer  program  or 
embedded  processor  software/firmware  which  prevents  or  precludes 
the  successful  performance  of,  or  accomplishment  of,  a  fimction  or  sub¬ 
function  as  specified  in  the  Development  Specification... and  the 
Program  Performance  Specifications...  [Ref.  53:  H-30] 

The  Performance  Warranty  Period  is  later  defined  as  a  period  of,  "two  years 
commencing  upon  the  acceptance  of  the  First  Article"  [Ref.  53;  H-31]. 

While  the  contract  also  includes  the  usual  special  provisions  regarding 
actions  that  result  in  the  warranty  being  "null  and  void",  and  the  administra¬ 
tion  of  such  a  warranty  is  going  to  be  a  great  management  challenge;  this 
type  of  agreement  is  definitely  a  step  in  the  right  direction.  [Ref.  54] 
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G.  LESSONS  LEARNED 


Here  are  a  few  quotes  that  have  been  extracted  from  respondents' 
answers  to  the  final  two  survey  questions  dealing  with  "lessons  learned"  and 
"other  concerns": 

•  Educate  both  the  customer  and  the  contractor,  during  post-award,  if  not 
before,  as  to  the  real  need  for  cooperation  (and)  understanding  of  the 
requirements 

•  Count  on  failure  if  your  contracting  officer  is  timid.  If  the  system  is  an 
infantry  system  —  make  the  PM  infantry.  If  the  system  is  for  armor  — 
make  the  PM  armor...  Keep  the  computer  wizards  out  of  the  way.  The 
systems  they  kno«v  work  at  the  speed  of  light,  but  they  don't. 

•  Don't  sleep  through  the  first  six  months  assuming  you  can't  get  in  much 
trouble  so  fast. 

•  Get  it  defined  up  front,  what  is  wanted,  when  its  wanted  and  how  much. 

•  Close  coordination  between  technical  and  contracting  personnel  during  the 
development  phase  to  assure  only  necessary  changes  are  made. 

•  1.  Analyze  the  contract  specification  and  develop  a  requirements 

document  which  identifies  what  you  will  implement  for  each  require 
ment  in  the  contract. 

2.  Review  your  "Requirements  Document"  with  the  government  so  that 
early  in  the  contract  they  will  know  how  you  have  interpreted  their 
requirements  and  how  you  intend  to  satisfy  each  one. 

3.  Identify  and  record  all  disagreements  and  resolve  them  immediately. 

4.  Then  lay  out  a  "doable"  schedule  and  go  to  work. 

Do  not  wait  until  a  CDR  to  tell  the  customer  what  he  will  receive  as  a 
product ! 

•  Never  assume  that  just  because  the  government  does  not  have  technically 
qualified  personnel  that  the  contractor's  personnel  are  more  qualified  than 
we  are. 

•  Implement  frequent  and  comprehensive  in  process  reviews.  Communi¬ 
cations  is  the  key. 

•  Management  is  illiterate  when  it  comes  to  software.  They  will  not  admit  it, 
therefore,  they  ignore  it. 

1.  Too  much  time  spent  studying 

2.  Too  little  time  spent  applying 

3.  Too  much,  "No,  you  can't  do  that" 

4.  Too  little,  "Tell  me  more” 


•  Realistic  time  and  cost  budgets  at  the  onset  are  essential  for  a  successful 
software  development  program 

•  Managing  software  development  is  a  fine  balance  between  strict  configu¬ 
ration  management  and  control,  and  the  freedom  to  let  the  designers  be 
innovative  and  be  personally  involved  in  their  work. 

•  Software  is  voodoo  to  most  people  and  it's  easy  to  get  suckered  (or  bulled) 
into  thinking  you  understand  all  the  requirements  of  the  technical  commu¬ 
nity. 

•  Contract  on  a  level  of  effort  basis  until  all  parties  agree  with  the  baseline 
requirements. 

•  An  ill  defined  project  cannot  be  monitored. 

•  It  is  very,  very  difficult  to  accurately  estimate  the  cost  and  schedule  for  a 
software  project  that  is  unprecedented.  Therefore  the  need  to  prototype 
difficult  areas  before  awarding  FSD  contract. 

•  A  customer  who  understands  his  requirement-  in  detail-  and  will  assist  in 
working  the  problem  is  irreplaceable. 

•  Risk  management  is  the  key  to  successful  development  programs 

•  Bright,  experienced  people  that  communicate  effectively  on  both  the 
Government  and  industry  teams  equals  success.  Drones  on  either  side,  or 
lack  of  communications,  ensures  failure. 

•  Identify  de  /elopment  problems  early.  Force  contractor  management  that 
meets  intermt,'’iate  objectives  and  milestones.  Documentation  is  the  most 
expensive  segmer.t  of  the  development. 

•  Well  defined  functional  requirements  are  imperative.  On-going  design 
review  is  critical.  Good  lines  of  communication  between  users,  managers, 
and  contractors  is  critical. 

•  Software  is  not  something  to  be  afraid  of  and  it  certainly  should  not  be 
ignored.  Many  times  software  is  ignored  or  played  down  due  to  sheer 
ignorance. 

H.  SUMMARY 

Survey  respondents  felt  that  nearly  60%  of  all  software  development 
programs  resulted  in  cost  overruns,  schedule  delays  and  poor  performance 
due  to  management  and/or  technical  issues.  They  felt  that  almost  47%  of  the 
development  programs  suffered  contractual  problem.  No  matter  where  the 
blame  lies,  it  is  clear  that  both  government  and  industry  personnel  have  the 
perception  that  a  problem  exists  in  the  government's  ability  to  procure 
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custom  designed  software  for  its  weapon  systems.  There  is  definitely  a 
perceived,  if  not  real,  problem  in  the  government's  ability  to  procure 
customed  designed  software.  Although  there  was  tremendous  diversity  in  the 
answers  to  questions  asked  during  the  data  collection  phase  of  this  research, 
one  underlying  fact  consistently  came  through  —  the  need  for  open  lines  of 
communication. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


"  IF  YOU  ALWAYS  DO 
WHAT  YOU  ALWAYS  DID 
YOU  WILL  ALWAYS  GET 
WHAT  YOU  ALWAYS  GOT  " 

CROMWELL  (CIRCA  1650) 
"AND  WE  DO.  WE  DO!" 

SURVEY  RESPONDENT  (1990) 


A,  CONCLUSIONS 

The  research  conducted  has  indicated  that  a  serious  problem  exists 
within  the  Department  of  the  Navy,  and  within  the  Department  of  Defense  in 
general,  concerning  the  procurement  of  customed  designed  software  to  be 
used  within  its  weapon  systems.  The  problem  was  first  brought  to  the  Gov¬ 
ernment's  attention  in  1979  by  the  General  Accounting  Office  in  its  report, 
"Contracting  for  Computer  Software  Development — Serious  Problems 
Require  Management  Attention  to  Avoid  Wasting  Additional  Millions."  In 
1987,  the  Defense  Science  Board  Task  Force  on  Military  Software  reported 
much  the  same  situation.  Their  report  to  the  Secretary  of  Defense  began: 

Many  previous  studies  have  provided  an  abundance  of  valid 
conclusions  and  detailed  recommendations.  Most  remain  unimple¬ 
mented.  If  the  military  software  problem  is  real,  it  is  not  perceived 
as  urgent.  We  do  not  attempt  to  prove  that  it  is;  we  do  recommend 
how  to  attack  it  if  one  wants  to.  [Ref.  47] 

Ten  years  after  the  original  GAO  report,  the  same  issues  regarding  the 

procurement  of  software  were  brought  to  the  attention  of  Congress  by 

their  own  Committee  on  Science,  Space  and  Technology  of  the  House  of 

Representatives. 
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As  discussed  earlier  in  Chapter  II,  the  major  Department  of  Defense 
directives  regarding  weapon  system  acquisition  do  not  recognize  the  critical 
importance  of  software  procurement.  This  importance,  and  even  more  so 
reliance,  on  embedded  software  deserves  special  recognition  in  directives  such 
as  DOD  Directive  5000.1,  Major  and  Non-major  Defense  Acquisition  Pro¬ 
grams,  and  DOD  Instruction  5000.2,  Defense  Acquisition  Program  Proce¬ 
dures.  Embedded  software,  by  the  sheer  volume  of  assets  consumed  in  its 
design,  development  and  maintenance,  warrants  special  attention  from  a 
resource  management  stand-point.  That  management  attention  will  occur 
only  when  the  major  DOD  weapon  system  acquisition  directives  specifically 
address  the  unique  requirements  associated  with  software  development.  The 
Department  of  Defense  Software  Master  Plan  is  a  giant  step  in  the  right 
direction.  More  recognition  must  follow. 

This  research  has  indicated  that  this  problem  is  recognized  not  only  by 
external  audit  agencies  such  as  the  General  Accounting  Office,  and  Congres¬ 
sional  committees,  but  also  by  the  staff  agencies  within  the  Department  of 
Defense  itself.  More  importantly,  the  problems  with  embedded  software  are 
recognized  by  both  the  government  and  industry  personnel  who  are  charged 
with  its  development,  procurement,  and  maintenance.  Three-fourths  of  the 
engineers,  contracting  personnel,  and  managers  who  responded  to  this 
research's  survey,  felt  that  at  least  one  half  of  the  software  development  pro¬ 
grams  resulted  in  cost  overruns,  schedule  delay  and  poor  performance  due  to 
program  management  issues.  They  all  recognized  that  software  development 
programs  typically  have,  in  addition  to  management  problems,  technical 
and/or  contractual  problems.  The  top  five  most  common  problems  identified 
by  survey  respondents,  (unclear  user  requirements,  too  many  changes  to 


63 


requirements,  lack  of  imderstanding  regarding  software  design  and  develop¬ 
ment,  schedule  too  optimistic,  and  unclear  statement  of  work),  all  point 
directly  at  management  issues  vice  problems  in  the  technical  or  contracting 
areas. 

How  can  a  program  office  expect  to  ultimately  satisfy  the  end  user's  mis¬ 
sion  need  when  that  critical  gap  between  the  user  and  the  software  engineer 
is  not  bridged?  This  research  has  shown  that,  more  than  any  other  problem, 
the  ability  to  accurately  and  correctly  identify  the  user's  requirements  is  the 
number  one  problem  with  software  development  today.  There  are,  however, 
current  development  practices  that  can  aid  in  building  that  bridge,  specifi¬ 
cally,  the  use  of  rapid  prototyping. 

Unfortunately,  it  is  not  an  easy  problem  to  correct.  The  heart  of  the  prob¬ 
lem  does  not  lie  in  the  complex  system  used  to  procure  software.  In  other 
words,  the  answer  to  embedded  software  development  is  not  in  the  contract¬ 
ing  mechanism  itself.  The  problem  is  fostered  by  the  attitude  and  manage¬ 
ment  decisions  of  the  personnel  within  that  procurement  system.  The  answer 
lies  in  changing  the  "attitude",  not  in  changing  the  basic  procurement  system. 
Our  program  managers  and  contracting  officers  must  learn  to  make  smart, 
well-informed  decisions  regarding  software  development. 

C.  RECOMMENDATIONS 

The  following  recommendations  are  a  result  of  this  research  effort: 

1.  Use  of  Rapid  Prototyping 

All  significant  software  development  programs  should  be  contractu¬ 
ally  structured  to  require  the  iterative  process  known  as  "rapid  prototyping." 
This  process  has  become  an  accepted,  and  in  most  cases,  the  preferred 
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method  of  development  to  ensure  the  ultimate  satisfaction  of  the  user's  needs. 
This  may  require  a  formal  policy  change. 

Four  of  the  top  five  areas  of  concern,  identified  by  the  survey 
respondents,  would  easily  have  less  of  an  impact  on  software  development 
programs  if  the  use  of  "rapid  prototyping"  became  the  standard  for  develop¬ 
ment.  Defining  the  user’s  requirements  would  become  an  iterative  process, 
thus  easing  the  pressure  on  the  program  manager  and  contracting  officer  to 
interpret  these  requirements  into  contractual  language  at  the  onset  of  the 
program.  User  involvement  throughout  the  phases  of  development  would  be 
assured.  This  would  also  ease  the  pressure  of  detailing  a  statement  of  work  in 
excruciating  detail  that  contracting  officers,  managers,  and  engineers  would 
be  forced  to  live  with  throughout  the  program.  Shorter  periods  of  perfor¬ 
mance,  through  the  use  of  intermediate  milestones  and  user  reviews,  would 
make  schedule  setting  more  realistic.  Finally,  the  impact  of  "too  many 
changes  to  requirements"  would  be  lessened  because  the  intent  of  the  proto¬ 
typing  effort  would  be  to  incrementally  massage  user  requirements,  taking 
the  appropriate  action  to  resolve  any  misinterpretations.  The  use  of  level  of 
effort  type  contracts  should  be  explored  as  a  possible  means  to  implement  this 
prototype  development  methodology. 

2.  Use  On-site  IV&V  on  all  Significant  Software  Development 

Efforts. 

The  use  of  independent  technical  and  management  teams  to  moni¬ 
tor  the  contractor's  processes  and  performance  is  not  an  inexpensive  under¬ 
taking.  The  benefits  from  budgeting  resources  at  the  front  end  of  a  program 
to  do  such  a  task,  however,  is  without  a  doubt  a  worthwhile  investment.  Two 
requirements  hold  constant  for  all  Independent  Verification  and  Validation 
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(IV&V)  efforts.  First,  the  program  office  must  be  willing  to  dedicate  the 
required  assets  to  the  IV&V  team,  basically  in  terms  of  appropriate  staffing 
for  the  assigned  task.  It  is  foolish  to  halfheartedly  provide  a  quasi-IV&V 
team.  In  the  end,  the  program  office  may  potentially  find  itself  in  twice  as 
much  trouble.  Secondly,  the  IV&V  team  must  be  competent  to  perform  the 
task.  Probably  the  most  obvious  requirement,  and  at  the  same  time,  the 
hardest  to  satisfy.  There  are  no  magical  answers  or  formulas  to  assess  the 
competency  of  an  IV&V  team.  This  is  made  even  more  difficult  because  like 
many  service  contracts  there  are  no  tangible  deliverable  items.  A  third 
requirement  which  is  almost  a  constant  is  the  need  for  the  IV&V  effort  to  be 
collocated  with  the  contractor.  It  is  absolutely  worth  the  time  and  money  to 
have  an  on-site  validation  and  verification  team  that  reports  directly  to  the 
program  office.  It  is  important  that  the  program  manager  not  get  lulled  into  a 
false  sense  of  security  by  using  an  IV&V  team.  Remember,  they  are  technical 
people  validating  the  technical  approaches  used  by  the  contractor.  Someone 
must  still  ensure  that  the  software  will  satisfy  the  user's  requirements. 

3.  Negotiate  Software  Performance  Warranties  in 

Development  Contracts 

Negotiating  a  performance  warranty  for  embedded  computer 
resources  may  be  somewhat  difficult,  at  best.  There  are  definitely  cost  versus 
benefit  trade-offs.  The  deliverable/operational  software  is  only  warranted 
against  defects  as  delineated  in  the  contract.  As  discussed  earlier,  a  standard 
measurement  of  software  quality  is  difficult  to  define.  The  contract  may 
therefore  call  for  the  contractor  produced  Product  Performance  Specification 
(PPS)  to  be  the  basis  for  the  warranty.  In  that  case,  the  government  is  imder 
the  gun  to  ensure  that  the  PPS,  in  fact,  satisfies  the  users  requirements. 
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Given  the  difficulties  described  above,  it  may  still  be  in  the  best  interest  of 
the  government  to  negotiate/require  warranties  for  embedded  software.  The 
more  common  they  become,  the  more  contractors  will  be  incentivized  to 
specifically  deliver  products  without  defects,  and  to  generally  improve  the 
overall  quality  of  their  software  development  process. 

4.  Education  of  Program  Managers  and  Contracting  Officers 
This  "attitude"  regarding  software  development  can  only  be 
changed  through  an  educational  process.  In  line  with  the  initiatives  to  man¬ 
date  educational  requirements  for  contracting  officers  and  program  man¬ 
agers,  a  look  at  courses  in  software  development  management  would  be 
appropriate.  The  answer  is  not  to  make  computer  programmers  contracting 
officers  and  program  managers.  It  goes  without  saying,  however,  to  make 
smart  decisions  management  personnel  must  be  somewhat  literate  with 
regard  to  software  development  techniques,  processes,  and  the  alternatives 
available.  One  way  to  aid  in  that  literacy  attainment  is  to  establish  specific, 
easily  accessible,  educational  courses  for  those  personnel  that  will  manage 
and  contract  for  software  development. 

D.  ANSWERS  TO  RESEARCH  QUESTIONS 

1.  What  currently  allowable  contractual  mechanisms  can  be 
used  to  improve  the  Department  of  the  Navy's  success  in  the 
procurement  of  custom  developed  software  ? 

As  discussed  in  the  conclusions  and  recommendations  to  this 

research  effort,  the  current  procurement  problems  regarding  customed 

designed  and  developed  software  can  all  be  addressed  using  management 

action  within  the  current  contracting  system.  The  use  of  prototyping  would 

require  a  re-thinking  of  the  way  we  normally  do  business,  but  it  is  not  only  a 
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viable  option,  it  is  becoming  the  most  valid  approach.  Acquisition  strategies 
should  call  for  the  use  of  an  on-site  IV&V  team  from  either  dedicated  in-house 
resources  or  procured  from  the  private  sector  though  a  service  type  contract. 
The  prime  contractor,  in  this  case  must  be  made  aware  of  the  IV&V's  rela¬ 
tionship  to  the  program  manager  and  contracting  officer,  and  the  extent  of 
their  authority.  The  last  contractual  mechanism  that  should  be  seriously  con¬ 
sidered  is  the  use  of  a  performance  warranty  to  both  protect  the  government 
and  incentivize  the  contractor. 

While  use  of  contracting  mechanisms  was  the  basic  premise  of  this 
research  effort,  it  became  obvious  that  the  problems  encountered  in  the  pro¬ 
curement  of  embedded  computer  resources  was  not  a  contracting  issue. 

2.  How  successful  is  the  Department  of  Defense/Department  of 

the  Navy  in  the  procurement  of  custom  software? 

Even  based  on  the  broad  definition  of  program  success  given  in 
Chapter  I,  this  question  remains  difficult  to  answer.  It  is  relatively  easy  to 
find  program  information  for  those  programs  that  have  suffered  setbacks. 
The  issue  gains  attention  because  of  the  nature  of  software  development.  It  is 
manpower  intensive,  extremely  costly,  and  delivers  no  tangible  product. 
When  setbacks  are  in  terms  of  billions  of  dollars  in  overruns,  years  of  slip¬ 
page  in  schedules,  and  deliver  of  software  that  is  not  safe  for  use,  the  crisis 
seems  phenomenal.  Unfortunately,  those  development  programs  that  are 
delivered  on  time,  on  budget,  and  that  work,  or  suffer  only  minimal  problems 
are  not  deemed  news  worthy  in  these  times  of  public  mistrust  of  government 
procurement.  Successful  programs  do  not  get  adequate  recognition  regardless 
of  the  product  being  developed. 
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3.  What  directives  regfuhkte  and/or  guide  Program  Managers 
and  Contracting  Officers  in  the  procurement  of  custom 
software  ? 

The  Brooks  Bill  (PL89-306)  of  1965,  and  the  Warner  Amendment  to 
the  Fiscal  Year  1982  Appropriations  Act  (PL  97-86) ,  are  the  two  specific  laws 
that  pertain  to  software  procurement.  They  mandate  a  clear  set  of  directions 
regarding  the  procurement  of  computer  hardware  and  software  for  adminis¬ 
trative  t3T)e  functions  and  military  missions,  respectively.  The  Federal 
Acquisition  Regulation  (FAR)  do  not  specifically  recognize  the  procurement  of 
software  as  being  different  from  the  procurement  of  any  other  good  or  service. 
The  DOD  FAR  Supplement  only  provides  guidance  as  far  as  recapitulating 
what  type  software  procurement  is  allowable  directly  by  the  DOD  under  the 
Warner  Amendment. 

Within  the  Department  of  the  Defense/Department  of  the  Navy 
there  are  three  major  directives  that  provide  guidance  regarding  the  pro¬ 
curement  and  management  of  embedded  computer  resources.  They  are:  DOD 
Directive  5000.29,  Management  of  Computer  Resources  in  Major  Defense 
Systems;  SECNAVINST  5200.32,  Management  of  ECR  in  Department  of  the 
Navy  Systems;  and,  MCO  5200.23A,  Management  of  Mission-Critical  Com¬ 
puter  Resources  (MCCR)  in  the  Marine  Corps.  Numerous  other  orders  and 
directives  pertaining  to  specific  areas  of  a  system  acquisition  such  as  test  and 
evaluation,  configuration  management,  and  standardization  are  also  applica¬ 
ble  to  embedded  computer  resource  development  and  procurement. 

There  are  two  military  standards  that  provide  the  majority  of 
the  specific  direction  to  the  program  manager  and  contracting  officer.  The 
first,  DOD-STD-2167A,  Defense  System  Software  Development,  delineates 
the  requirements  for  mission  critical  software  development.  The  second. 
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DOD-STD-2168,  Defense  System  Software  Quality  Program,  establishes  the 
requirements  for  a  contractor’s  software  quality  assurance  program.  These 
two  standards  are  designed  to  be  used  in  conjunction  with  each  other. 

4.  What  contract  tjrpes  are  typically  used  for  programs 
involving  the  development  of  custom  software  ? 

There  is  no  one  contract  type  can  be  identified  as  "typically  used" 

for  software  development.  More  importantly,  there  is  no  one  contract  type 

that  can  be  identified  as  being  the  best  for  software  development.  The  full 

spectrum  of  contract  types,  from  Firm-Fixed-Price  to  Cost- Plus- Award-Fee 

contracts,  were  reported  as  having  been  used  by  the  survey  respondents.  The 

research  showed  that  every  type  contract  has  the  potential  for  being  used, 

and  that  the  selection  is  totally  dependent  upon  the  situation. 

5.  Are  there  any  contractual  mechanisms  that  can  be  used  to 
aid  in  the  development  of  custom  software  ? 

The  provisions  of  DOD  Standards  2167A  and  2168  when  called  out 
in  a  contract  are  probably  the  best  contractual  direction  that  we  can  provide  a 
contractor  during  a  software  development  program.  The  use  of  IV&V  teams 
and  performance  warranties  will  help  to  ensure  the  contractor  is  number  one 
doing  his  job,  and  number  two,  willing  to  back  the  quality  of  his  work.  Most  of 
the  other  influences  pertaining  to  software  development  contract  are  man¬ 
agerial  in  nature.  In  other  words,  the  government  needs  program  managers 
and  contracting  officers  who  can  make  good  business  decisions  while  satisfy¬ 
ing  the  user's  mission  requirements. 


E.  RECOMMENDATIONS  FOR  FURTHER  STUDY 

1.  Formal  Software  Development  Education 

Examine  the  existing  management  schools  within  the  Department 
of  Defense  to  evaluate  the  content  and  the  adequacy  of  courses  offered  dealing 
with  the  management  and  procurement  of  custom  developed  software.  This 
research  effort  indicated  that  the  root  of  the  software  development  problem  is 
not  in  the  procurement  system  itself,  but  in  the  personnel  within  the  system. 
As  the  dependence  on  software  continues  to  grow,  the  ability  of  goverrunent 
procurement  personnel  to  understand  the  nature  of  design  and  development 
of  mission  critical  software  will  become  increasingly  more  critical. 

2.  Use  of 'Rapid  Prototyping"  in  Software  Development 
Examine  various  means  to  implement  "rapid  prototyping"  as  a 

standard  development  process  for  mission  critical  computer  resources.  This 
would  involve  an  investigation  of  various  contracting  strategies  that  would 
support  the  use  of  software  prototyping  in  development  programs  for  weapon 
systems.  As  discussed  above,  the  use  of  prototyping  would  go  a  long  way  in 
helping  to  resolve  four  of  the  five  top  software  development  issues  identified 
in  this  research.  In  conjunction  with  the  examination  of  contracting  tech¬ 
niques,  a  review  of  existing  policies  dealing  with  the  use  of  prototyping  as  an 
acquisition  strategy  should  be  undertaken.  It  may  be  useful  to  examine  any 
programs  that  have  used  prototjrping  techniques  for  embedded  software 
development,  and  develop  a  model  program  for  software  procurement  based 
on  this  approach. 

3.  Use  of  Level  of  Effort  Contracts  in  Software  Development 
Examine  the  use  of  level  of  effort  type  contracts  for  software  devel¬ 
opment.  In  line  with  the  recommendation  above,  one  way  of  emulating  a 
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prototype  approach  is  to  use  progressive  level  of  effort  contracts,  and/or 
competitive  level  of  effort  contracts.  The  Fleet  Combat  Direction  System 
Support  Activity  at  Dam  Neck,  VA,  is  doing  just  that.  This  research  effort 
would  include  an  analysis  of  their  procedures  for  procuring  custom  software 
in  the  development  phase,  the  production  phase,  and  life  cycle  maintenance 
phase  for  a  weapon  system.  [Ref.  55] 

4.  Software  Performance  Warranties 

Examine  the  value  of  software  performance  warranties  in  weapons 
systems  development  and  procurement  contracts.  Although  the  AN/TYQ-23, 
Tactical  Air  Operations  Module,  production  contract  has  provisions  that  call 
out  a  two  year  software  performance  warranty,  the  system  has  yet  to  be 
fielded.  The  warranty  is  based  on  the  system’s  compliance  with  the  software 
Product  Performance  Specification.  Looking  back  to  Figure  4.3,  one  can  see 
the  relative  cost  of  software  changes  that  would  be  required  to  correct  defi¬ 
ciencies.  Therefore,  one  can  easily  deduce  that  the  implementation  of  the 
warranty  clause  will  be  difficult,  at  best.  Questions  such  as:  in  scope/out-of¬ 
scope?;  system  operated  in  accordance  with  operating  manuals?;  proper  doc¬ 
umentation  of  defects,  and  ability  to  recreate  the  defect?;  will  be  negotiated 
well  beyond  the  warranty  period.  This  recommended  research  would  analyze 
the  effectiveness  of  those  systems  acquisition  contracts  that  contain  software 
performance  warranties.  Another  approach  would  be  to  produce  a  model  or  a 
checklist  for  use  as  a  guide  when  negotiating  software  warranties. 

6.  Model  for  a  Weapon  System  Incentive  Contract 

Develop  a  model  incentive  type  contract  that  can  be  tailored  and 
used  for  weapon  system  procurement  involving  both  hardware  and  software 
development.  During  the  acquisition  strategy  formulation  stages,  it  would  be 
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extremely  helpful  to  have  a  straw-man  or  model  incentive  contract  to  aid  in 
the  structuring  of  the  contractual  mechanism  that  would  focus  both  hardware 
and  software  criteria.  The  ability  and  practicality  of  segregating  software 
from  hardware  criteria  would  need  to  be  investigated.  The  practicality  and 
economics  of  using,  for  example,  an  award  fee  contract  with  incentives  for 
software,  as  well  as,  hardware  accomplishments  would  require  analysis.  This 
type  approach  would  accomplish  two  valuable  objectives.  First,  the  contractor 
would  be  monetarily  incentivized  to  place  more  attention  on  those  software 
criteria  deemed  critical  by  the  program  office.  Second,  the  program  office 
would  be  forced  (by  the  contractor  seeking  his  monetary  award)  to  pay  closer 
attention  to  software  development  issues.  This  effort  could  also  investigate 
the  establishment  of  a  model  for  the  award  criteria  itself.  In  other  words,  it 
would  research  current  software  measures  of  effectiveness  that  could  be 
appropriately  tied  to  an  incentive  type  contract. 
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APPENDIX  A 


SURVEY  QUESTIONNAIRE 

CONTRACTING  FOR  CUSTOM  DEVELOPED  SOFTWARE  WITHIN  DOD 

1 .  What  is  your  most  recent  job  experience? 


A.  Government 

Industry 

B.  Program  Manager 

Program  Engineer 

Contracting  Officer  (PCO) 

DCAS  QA  Rep 

Contract  Administrator 

Tech  Rep 

Project  Officer 

Other 

(specify) 

2.  Have  you  been  involved  in  a  program  that  required  the  development  of  custom  software  (i.e.,  major 
weapon  system)  for  the  Department  of  Defense? 

Yes _  No _  If  YES  please  CONTINUE — If  NO,  STOP  and  kindly  return  the  survey. 

3.  How  many  years  experience  do  you  have  with  acquisition  programs  requiring  custom  software? _ 

4.  Where  would  you  say  that  ihe  majority  of  your  expertise  lies? 

Program  Manager  _  Program  Engineer  _ 

Contracting  Officer  (PCO) _  DCAS  QA  Rep  _ 

Contract  Administrator  _  Tech  Rep  _ 

Project  Officer  _  Other _ 

(specify) 

The  following  three  questions  deal  with  problems  regarding  computer  software  within  a  complex  weapon 
system.  For  each  question,  place  an  'x’  along  the  corresponding  scale. 


5.  Do  technical  developmental  problems  (i.e.,  programming/interface  difficulties)  lead  to  cost  overruns, 
schedule  delays  and  poor  performance? 


_ I _ 1  _ 1 _ L 

100% 

I 

75% 

50% 

25% 

r 

0% 

Almost  Always  Sometimes  Very  Rarely 


6.  Do  contracting  related  problems  lead  to  cost  overruns,  schedule  delays  and  poor  performance? 


1 _ 1 _ 1  _ 1 _  _l 

n 

1 00% 

75% 

50% 

25% 

r 

0% 

Almost  Always  Sometimes  Very  Rarely 
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7.  Do  program  management  related  problems  lead  to  cost  overruns,  schedule  delays  and  poor 
performance? 


100%  75%  50% 

Almost  Always  Sometimes 


25%  0% 

Very  Rarely 


8.  Please  rank  the  FIVE  (1  thru  5)  most  common  problems  regarding  contracting  for  and  the  manage¬ 
ment  of  a  program  involving  software  development  (with  1  being  the  most  common  problem) 


Unclear  user  requirements 
Schedule  too  optimistic 
Lack  of  govl  regulations 
Inadequate  govt  specifications 
Lack  of  timely  user  feedback 
Wrong  personnel  at  design  reviews 
Lack  of  software  design  freeze 
Failure  to  designate  POCs 
Improper  contract  type 
Lack  of  qualified  goVt  tech  personnel 
Too  many  changes  to  requirements 
inability  to  estimate  cost 
Lack  of  understanding  regarding 
software  design  &  development 
Mismanagement  by  Industry 
Others _ 


Unclear  Statement  of  Work 
Contract  clauses  too  restrictive 
Too  many  govt  regulations 
Too  many  govt  specifications 
Ladt  of  design  reviews 
Lack  of  hardware  design  freeze 
Inadequate  testing  (govt) 
inadequate  testing  (contractor) 
Difficult  acquisition  procedures 
Inherent  inability  to  measure 
reliability  of  software 
Lack  of  contractor  incentive 
Lack  of  adequate  documentation 
Lack  of  configuration  mgmt 
Mismanagement  by  govt 


ADDITIONAL  SPACE  IS  REQUIRED  FOR  ANY  OF  THE  FOLLOWING  QUESTtONS.  PLEASE  ATTACH 
A  SEPARATE  SHEET  OF  PAPER.  _HAND  WRITTEN  CONTRIBUTIONS  ARE  FINE. 


9.  Expand  your  thoughts  and  experiences  regarding  the  too  three  items  that  you  chose  in  the  preceding 
question. 

#1  _ 


#2 
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1 0.  What  do  you  feel  is  the  most  important  ingredient  to  the  success  of  a  software  development  phase  in 
a  major  weapon  system  procurement  from  a  contracting  aspect? 


1 1 .  What  do  you  feel  is  the  most  important  ingredient  to  the  success  of  a  software  development  phase  in 
a  major  weapon  system  procurement  from  a  management  aspect? 


1 2.  What  type  contracts  have  you  worked  with  on  software  development  programs? 

Firm  Fixed  Price _  Fixed  Price  Incentive _  Cost  Plus  Fixed  Fee _  Cost  Plus  Award  Fee 

Other _ _ _ _ 


1 3.  Have  you  experienced  any  particular  benefits  or  drawbacks  associated  with  any  of  the  contract  types 
mentioned  in  question  12  above? 


1 4.  What  contract  type  do  you  feel  offers  the  most  benefit  to  both  the  government  and  the  contractor 
during  a  software  development  effort? 
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What  is  the  most  important  lesson  you  have  learned  (from  either  a  good  or  bad  experience)  in  dealing 
with  a  contract  that  requires  software  development? 


Oo  you  have  any  othar  concerns  regarding  the  procurement  of  custom  developed  software? 


THANK  YOU  FOR  YOUR  HELP  AND  COOPERATION 


APPENDIX  B 


SURVEY  DATA 


NO, 

GOVT 

INDUSTRY 

POSITION 

YBS 

lESh 

CONTRACT 

MGMT 

1 

X 

SQA  SUPERVISOR 

4 

80 

50 

50 

2 

X 

CX)NTRACT  ADMIN 

2 

70 

25 

35 

3 

X 

GOVT  CXDNTRACTS  MGR 

10 

20 

70 

85 

4 

X 

PROGRAM  ENGINEER 

3 

80 

60 

60 

5 

X 

CONTRACT  ADMIN 

5 

50 

50 

50 

6 

X 

CONTRACT  ADMIN 

8 

100 

75 

25 

7 

X 

PCO 

8 

45 

15 

55 

8 

X 

PROGRAM  MANAGER 

30 

80 

65 

50 

9 

X 

TEST  MGR 

3 

80 

80 

50 

10 

X 

CONTRACT  ADMIN 

5 

85 

70 

50 

11 

X 

PROGRAM  ENGINEER 

8 

50 

25 

65 

12 

X 

PCO 

4 

75 

70 

50 

13 

X 

TEST  DIRECTOR 

8 

75 

20 

70 

14 

X 

PROGRAM  ENGINEER 

22 

20 

20 

20 

15 

X 

PROGRAM  MANAGER 

22 

90 

65 

90 

16 

X 

DCASPRO  ENGINEER 

5 

20 

15 

15 

17 

X 

COMPUTER  ENGINEER 

15 

70 

85 

55 

18 

X 

PROGRAM  ENGINEER 

2 

70 

65 

65 

19 

X 

PROGRAM  ENGINEER 

7 

80 

80 

35 

20 

X 

TECHNICAL  REP 

10 

100 

65 

90 

21 

X 

DCAS  (DCMR)  ENGINEER 

10 

50 

65 

50 

22 

X 

CONTRACT  ADMIN 

8 

35 

50 

75 

23 

X 

SQA 

13 

60 

50 

75 

24 

X 

SQA 

7 

75 

70 

85 

25 

X 

PROGRAM  MANAGER 

7 

30 

20 

40 

26 

X 

PROGRAM  MANAGER 

25 

75 

25 

45 

27 

X 

CONTRACT  ADMIN 

3 

45 

60 

65 

28 

X 

TECHNICAL  REP 

10 

75 

50 

65 

29 

X 

PROGRAM  ENGINEER 

13 

75 

75 

75 

30 

X 

PROGRAM  ENGINEER 

5 

45 

90 

80 

31 

X 

QAREP 

10 

40 

20 

45 

32 

X 

PROGRAM  ENGINEER 

20 

50 

35 

75 

33 

X 

PROGRAM  ENGINEER 

23 

15 

25 

10 

34 

X 

PROGRAM  MANAGER 

12 

80 

50 

75 

35 

X 

V.P.  C3  SYSTEMS 

20 

40 

70 

70 

36 

X 

PROGRAM  MANAGER 

7 

75 

25 

50 

37 

X 

PROJECT  OFFICER 

3 

75 

50 

100 

38 

X 

SYS  ANALYST 

2 

75 

20 

65 

39 

X 

PROJECT  OFFICER 

1 

50 

25 

25 

40 

X 

SOFTWARE  ENGINEER 

4 

65 

70 

60 

41 

X 

PROJECT  OFFICER 

4 

80 

50 

50 

42 

X 

COTR 

4 

75 

25 

25 

43 

X 

PROGRAM  MANAGER 

10 

50 

20 

75 

44 

X 

QAREP 

17 

20 

0 

90 

45 

X 

PROGRAM  MANAGER 

8 

25 

40 

50 

46 

X 

PROGRAM  MANAGER 

10 

30 

25 

60 

47 

X 

CONTRACT  OFFICER 

14 

50 

20 

40 

48 

X 

COMPUTER  ENGINEER 

12 

20 

40 

65 

49 

X 

PROJECT  OFFICER 

5 

80 

45 

80 

78 


TECHNICAL 


NO. 

COVT 

INDIiSIRY 

EQSaiQN 

TECH 

CONTRACT 

MGMT 

15 

X 

COMPUTER  ENGINEER 

70 

85 

55 

12 

X 

COMPUTER  ENGINEER 

20 

40 

65 

10 

X 

DCAS  (DCMR)  ENGINEER 

50 

65 

50 

5 

X 

DCASPRO  ENGINEER 

20 

15 

15 

23 

X 

PROGRAM  ENGINEER 

15 

25 

10 

22 

X 

PROGRAM  ENGINEER 

20 

20 

20 

20 

X 

PROGRAM  ENGINEER 

50 

35 

75 

13 

X 

PROGRAM  ENGINEER 

75 

75 

75 

8 

X 

PROGRAM  ENGINEER 

50 

25 

65 

7 

X 

PROGRAM  ENGINEER 

80 

80 

35 

5 

X 

PROGRAM  ENGINEER 

45 

90 

80 

3 

X 

PROGRAM  ENGINEER 

80 

60 

60 

2 

X 

PROGRAM  ENGINEER 

70 

65 

65 

17 

X 

QAREP 

20 

0 

90 

10 

X 

QAREP 

40 

20 

45 

4 

X 

SOFTWARE  ENGINEER 

65 

70 

60 

13 

X 

SOA 

60 

50 

75 

7 

X 

SQA 

75 

70 

85 

4 

X 

SQA  SUPERVISOR 

80 

50 

50 

2 

X 

SYS  ANALYST 

75 

20 

65 

10 

X 

TECHNICAL  REP 

100 

65 

90 

JiL_ 

X 

TECHNICAL  REP 

_1£ _ 

.  _5Q _ 

_65 _ 

10,1 

AVERAGES 

56.14 

48.9 

58.9 

CONTRACTING 


NO. 

SOYT 

INDUSTRY 

POSITION 

TECH 

CQNIBACJ 

MgMI 

8 

X 

CONTRACT  ADMIN 

100 

75 

25 

8 

X 

CONTRACT  ADMIN 

35 

50 

75 

5 

X 

CONTRACT  ADMIN 

85 

70 

50 

5 

X 

CONTRACT  ADMIN 

50 

50 

50 

3 

X 

CONTRACT  ADMIN 

45 

60 

65 

2 

X 

CONTRACT  ADMIN 

70 

25 

35 

14 

X 

CONTRACT  OFFICER 

50 

20 

40 

4 

X 

COTR 

75 

25 

25 

10 

X 

GOVT  CONTRACTS  MGI 

20 

70 

85 

8 

X 

PCO 

45 

15 

55 

4 

X 

PCO 

_Z5 _ 

_Z2 _ 

_5J2 _ 

6.45 

AVERAGES 

59.09 

48.18 

50.46 

CONTRACT 


CPAF 

CPFF 

CPAF 


CPFF 


CPAF 


FPl 

FPI 

CPAF 

FPI 

CPFF 

FFP 

FPI 


CONTRACT 

CPIF 

FPI 

CPAF 

COST  PLUS 

CPAF 

CPAF 

COST  TYPE 
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MANAGEMENT 


NO.  GOVT 

INDUSTRY 

poamoN 

TECH 

CONTRACT 

MGMT 

CONTRACT 

15  X 

COMPUTER  ENGINEER 

70 

85 

55 

30 

X 

PROGRAM  MANAGER 

80 

65 

50 

CPFF 

25  X 

PROGRAM  MANAGER 

75 

25 

45 

CPIF 

22  X 

PROGRAM  MANAGER 

90 

65 

90 

12 

X 

PROGRAM  MANAGER 

80 

50 

75 

CPIF 

10  X 

PROGRAM  MANAGER 

50 

20 

75 

10 

X 

PROGRAM  MANAGER 

30 

25 

60 

8 

X 

PROGRAM  MANAGER 

25 

40 

50 

CPAF 

7  X 

PROGRAM  MANAGER 

75 

25 

50 

CPFF 

7  X 

PROGRAM  MANAGER 

30 

20 

40 

FPI 

5  X 

PROJECT  OFFICER 

80 

45 

80 

FPI 

4  X 

PROJECT  OFFICER 

80 

50 

50 

CPAF 

3  X 

PROJECT  OFFICER 

75 

50 

100 

FFP 

1  X 

PROJECT  OFFICER 

50 

25 

25 

8  X 

TEST  DIRECTOR 

75 

20 

70 

FPI 

3 

X 

TEST  MGR 

80 

80 

50 

CPFF 

_2Q_ 

_4Q _ 

.  _2Q _ 

_ 

10.9 

AVERAGES 

63.44 

42.19 

61.25 

PERCENTAGE  OF  SOFTWARE  DEVELOPMENT  PROGRAMS 
THAT  SUFFER  PROBLEMS  AS  A  RESULT  OF  TECHNICAL, 
CONTRACT,  OR  MANAGEMENT  ISSUES 


PERCENTAGE  OF  PROGRAMS 


RESPONDENT 

TECHNICAL 

CONTRACT 

MANAGEMENT 

DCAS  SQA 

80 

50 

50 

COMMANDER  DCASPRO 

70 

30 

40 

PROCUREMENT  MANAGER 

20 

70 

85 

PROGRAM  ENGINEER 

80 

60 

60 

CONTRACT  ADMINISTRATOR 

50 

50 

50 

CONTRACT  ADMINISTRATOR 

100 

75 

25 

CONTRACTING  OFFICER 

50 

10 

55 

PROGRAM  ENGINEER 

80 

65 

50 

TEST  DIRECTOR/MANAGER 

85 

80 

50 

CONTRACT  ADMINISTRATOR 

85 

70 

50 

SYSTEMS  ENGINEER 

50 

25 

70 

COMMANDER  DCASPRO 

75 

70 

55 

TEST  DIRECTOR 

75 

20 

70 

NAVPRO  ENGINEER 

35 

50 

75 

DCAS  ENGINEER 

50 

65 

50 

PROGRAM  MANAGER 

30 

20 

45 

PROGRAM  MANAGER 

75 

25 

40 

COMPUTER  ENGINEER 

65 

85 

55 

PROGRAM  ENGINEER 

70 

65 

65 

PROGRAM  EilGINEER 

80 

80 

35 

PROGRAM  MANAGER 

90 

65 

90 

CONTRACT  MANAGER 

45 

65 

60 

V.P.  C3I  SYSTEMS 

40 

75 

75 

TECH  REP 

75 

50 

70 

PROGRAM  ENGINEER 

75 

75 

75 

PROGRAM  ENGINEER 

45 

85 

80 

QAREP 

40 

20 

45 

PROGRAM  ENGINEER 

20 

20 

20 

PROGRAM  ENGINEER 

50 

40 

75 

PROGRAM  ENGINEER 

15 

25 

5 

PROGRAM  ENGINEER 

85 

50 

75 

DCASPRO  ENGINEER 

15 

15 

15 

PROGRAM  MANAGER 

50 

35 

75 

PROJECT  OFFICER 

50 

25 

25 

PROGRAM  MANAGER 

75 

20 

65 

CONTRACT  ADMINISTRATOR 

75 

50 

50 

PROGRAM  MANAGER 

75 

25 

50 

PROJECT  OFFICER 

75. 

50 

100 

AVERAGE 

61.83 

49.88 

57.80 
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RANKING 


MOST  COMMON  PROBLEM  IN 
SOFTWARE  DEVELOPMENT 


WT. 

RESPONSE 


1  Unclear  user  requirements  252.0 

2  Too  many  changes  to  requirements  1 40.0 

3  Lack  of  understanding  regarding  software  design  1 34.0 

4  Schedule  too  optimistic  130.0 

5  Unclear  statement  of  work  92.0 

6  Lack  of  (^alified  gov1  tech  personnel  88.0 

7  Inadequate  govi  specifications  76.0 

8  Lack  of  software  design  freeze  66.0 

9  Inability  to  estimate  cost  54.0 

10  Lack  of  adequate  documentation  46.0 

1 1  Mismanagement  by  the  government  40.0 

12  Difficult  acquisition  procedures  36.0 

13  Inadequate  testing  (contractor)  28.0 

14  Mismanagement  by  Industry  28.0 

1 5  Lack  of  timely  user  feedback  26.0 

16  Lack  of  configuration  mgmt  26.0 

17  Lack  of  contractor  incentive  26.0 

18  Lack  of  hardware  design  freeze  24.0 

19  Wrong  personnel  at  design  reviews  24.0 

20  Too  many  gov1  regulations  24.0 

21  Improper  contract  type  16.0 

22  Inability  to  measure  reliability  of  software  16.0 

23  Too  many  govl  specifications  12.0 

24  Contract  clauses  too  restrictive  lO  .O 

25  System  Engineering  failures  10.0 

26  Inadequate  source  selectbn  10.0 

27  Inadequate  testing  (gov1)  6,0 

28  Lack  of  design  reviews  4.0 

29  Failure  to  designate  POCs  2.0 

30  Inability  to  measure  software  maintainability  2.0 

31  Lack  of  gov't  regulations  0.0 
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APPENDIX  C 


LIST  OF  PERSONNEL  INTERVIEWED 


1.  Carlisle,  Jimmie.  Technical  Manager,  Defense  Systems  Division, 
Atlantic  Research  Corporation,  Van  Nuys,  California,  Interview,  April 
1990. 

2.  Bosworth,  Ted.  Vice  President  for  C3  Systems,  COMPUTER  Research 
Inc.,  Buffalo,  New  York,  Interview  ,  May  1990. 

3.  Firestone,  David.  Acquisition  Engineer,  AN/TYQ-23  Tactical  Air  Opera¬ 
tions  Module,  Space  and  Naval  Warfare  Command,  Washington,  D.C., 
Interview,  March  1990. 

4.  Mrazik,  David  R.  CWO-3  USMC.  Software  Quality  Assurance  Officer, 
Marine  Corps  Tactical  Systems  Support  Activity,  Camp  Pendleton.  Cali¬ 
fornia,  Interview,  April  1990. 

5.  Primm,  Ed.  Fleet  Combat  Direction  System  Support  Activity,  Dam 
Neck,  Virginia,  Interview,  May  1990. 

6.  Quattlebaum,  John  R.  Lt  Col  USMC.  Deputy  Program  Manager, 
AN/TYQ-23  Tactical  Air  Operations  Module,  Space  and  Naval  Warfare 
Systems  Command,  Washington,  D.C.,  Interviews,  January  and  March 
1990. 

7.  Swizewski,  James.  Contracting  Officer,  Naval  Regional  Contracting  Cen¬ 
ter,  Philadelphia,  Pennsylvania,  Interview,  March  1990. 

8.  Thomas,  Bruce  J.  Major  USMC.  Naval  Electronic  Systems  Command 
Technical  Representative,  Litton  Data  Systems,  Van  Nuys,  California, 
Interview,  April  1990. 

9.  Wasgatt,  Bud.  Division  Head,  Combat  Systems  Test  and  Analysis 
Department,  Naval  Ship  Weapon  Systems  Engineering  Station,  Port 
Hueneme,  California,  Inteview,  April  1990. 
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